Patrick Galbraith ([info]capttofu) wrote,
@ 2008-02-01 20:28:00
Previous Entry  Add to memories!  Share  Next Entry
Entry tags:federated, mysql, storage engines

FederatedX Pluggable Storage Engine Released!
The initial release of the FederatedX Pluggable Storage Engine for MySQL is now
available. I developed the Federated Storage Engine when I worked at MySQL,
and really saw a lot of potential with it. However, there were many other
projects that I had to give attention to, and many features and bug fixes that
users wanted didn't come into fruition because of the busy schedule. I left
MySQL a year ago to pursue an opportunity with Grazr what has kept me equally
busy. However, I have made a resolution this year to give some projects that I
have wanted to improve the attention that they need. I still see a lot of
potential with the "federated" concept.

Federated as it is isn't what many people expect it to be. IBM for instance,
has a federation as a very integral part of DB2. The Federated Storage Engine
is a proof-of-concept storage engine -- not to say it doesn't have
functionality that people can and are using currently.

I'd like to do what I can to change that. Some enhancements that I can think
of are:


  • Push-down conditions - being able to have a LIMIT be pushed down
    to a remote data source so a user isn't hammered with a huge data set.

  • Better transactional functionality. There is a patch out there that is
    suffering bit-rot that I would like to get included.

  • Other connection protocols. I wrote the Federated ODBC Storage Engine (I
    will release this soon too after I clean it up and test it. It would be good however to have a layer for other connections that doesn't require a separate storage engine. Also think JDBC, native client libs for other RDBMSs (think of the potential for migration tools!)

  • Handling large data sets - there must be a better way!.


Releasing this as a pluggable storage engine gives it independence from the
main server to facilitate faster improvement through more frequent bug fixes
and enhancements (patches, patches, and also patches!)

This release of FederatedX Pluggable Storage Engine is a fork of the Federated Storage Engine as it currently is, but that will change as it is modified.

About FederatedX (for those who haven't used Federated before)

FederatedX Plug-able Storage Engine is a storage engine for MySQL that allows you to create a table which uses a network connection to a remote database table as its data source, as opposed to most storage engines that use a table file or table-space file as a data source.

The source is available from:

http://hg.patg.net/federatedx-storage-engine/

tar.gz:

http://patg.net/federatedx_engine-0.1.tar.gz


(7 comments) - (Post a new comment)

Re pushing down LIMIT
(Anonymous)
2008-02-05 04:34 am UTC (link)
Push-down conditions - being able to have a LIMIT be pushed down
to a remote data source so a user isn't hammered with a huge data set.


We're working on something similar for Falcon, I *think* API support passing n_needed_records down to storage engines (and the SQL layer part indeed) will be ready sometime this Spring.

Best,
Sergey Petrunia, sergefp {at} mysql.com

(Reply to this) (Thread)

Re: Re pushing down LIMIT
[info]capttofu
2008-02-05 02:39 pm UTC (link)
Hi Sergey!

Thanks for that info. Having worked on the Falcon team, I had forgot that LIMIT wasn't pushed down to Falcon either. I do remember that just as Federated does, Falcon engine requires SQL being built and sent to the underlying database engine. Is falcon currently using pushdown for other parts of the query such as non-indexed columns? The problem I had in my pushdown patch is that if you performed a join with another table, the pushdown conditions would include parts of the query referencing the other table, which would not work with federated because you only want to pushdown the remote table. The same thing would hold true with Falcon as well since the join could reference a non-falcon table.

BTW - I would be glad to work out getting patches/bug fixes to fix bugs for MySQL Federated.

(Reply to this) (Parent)

How to Install
[info]jbn.pip.verisignlabs.com
2008-10-07 02:47 pm UTC (link)
Patrick,

I've been a long time user of the Federated engine and am really disappointed by it's performance issues. I'd like to change to FederatedX.

Unfortunately, I don't understand how to install it. The .tar.gz link above does not work. The other link only has individual files. There is no explanation about how to incorporate this source into an existing MySQL server installation.

Do you have any info?

Thanks,
Justin

(Reply to this) (Thread)

Re: How to Install
[info]jbn.pip.verisignlabs.com
2008-10-07 03:16 pm UTC (link)
Patrick,

Never mind. I finally found a good link via :
http://capttofu.livejournal.com/7291.html

(Reply to this) (Parent)

FederatedX on MySQL 5.1?
[info]jbn.pip.verisignlabs.com
2008-12-23 05:41 pm UTC (link)
Patrick,

Have you had a chance to resolve the issue with FederatedX on MySQL 5.1?

I REALLY like federated tables, but the performance issues are killing me.

Thanks,
Justin Noel
AppBeacon.com

(Reply to this)

Use with OpenLDAP?
[info]dclark.id.fsf.org
2009-03-31 05:22 pm UTC (link)
We currently have hashed passwords in a MySQL database and use a non-standard mechanism to allow several of our web sites to authenticate against this database.

We want to move to using OpenLDAP as the cannonical storage space for authentication information (and probably eventually OpenID as the single sign-on system), but are faced with the problem of keeping everything working and in sync during the (long) migration period.

Would moving a table of hashed MD5 passwords out of a MySQL database into OpenLDAP and then using FederatedX to query OpenLDAP (transparently to the database applications that expect the information to be in MySQL) be the kind of thing you think FederatedX would be good for?

Thanks,
--
Daniel JB Clark | Sys Admin, Free Software Foundation
pobox.com/~dclark | http://www.fsf.org/about/staff#danny

(Reply to this)


[info]chenjiayuan
2010-12-09 01:19 am UTC (link)
Do you like shanghai escort service providing elite beijing escort.Our online kitchen cabinets wholesale store supply discount kitchen and Bathroom Cabinet at wholesale prices.Buy Replica louis vuitton bags,We offer chanel bags, discount replica handbags outlets. 搬家团队非常盛行,杭州三替三塘搬家公司是经工商局注册的杭州搬家服务专业性一家专业性的杭州搬家公司.凌宇美国投资移民 LingYu American investment immigrant.资深网站google优化团队拥有资深的网站推广方面搜索引擎优化seo服务搜索一定好,技术方面Google推广Google promotion功能,还对海外方面进行海外推广Overseas promotion活动,让客户在google搜索上有好的Google排名Google rank非常好.

(Reply to this)


(7 comments) - (Post a new comment)

Image by [info]jussy_baby. Join the contest in [info]remixed!
Create an Account
Forgot your login or password?
Facebook Twitter More login options
English • Español • Deutsch • Русский…