Hi! It is about time to write some thoughts about Drizzle, even after it got so much of blogging love elsewhere :)
I love some of the ideas – like employing generic portable record format, and throwing away lots and lots of crufty code associated with reading internal structures.
Some of the ideas I probably love less, mostly the microkernel design.
See, I’m a believer in hacks. A hack in monolith code blends in nicely, a hack in microkernel design looks like bunch of spaghetti on top of kosher pork steak (well, probably bad analogy :). Hacks start bloating the plugin interfaces, microkernel designers become unhappy, there’s lots of tension, instead of living in one huge nice pot of spaghetti.
Why does one need hacks? I like when InnoDB controls the replication (thus adding transactional consistency to it, or adding semi-sync properties), and I like when replication controls the InnoDB (asks for higher priorities, and such). These changes required changing handler interface without even having replication as a module. In case of spaghetti soup, one straw more or less, doesn’t matter that much. :)
The very example of Apache proves the point, that modules don’t work well together. There’s not that much synergy between, say, mod_php and mod_perl. Actually, there’s not much synergy between any Apache module. People end up compressing, logging, filtering, redirecting inside PHP or Python or Perl code, not dedicated Apache modules. Why? Simply, because interfaces are insufficient, and modules end up limited – there’s no real synergy out there. In the end, having data logic in one piece is actually more maintainable than building bridges between entirely separate logic pools.
It is a bit of hypocrisy to aim for modular design with clear plugin interfaces, and at the same time remove all the features, that make design of other applications more modular and using clear interfaces (SPs, prepared statements, triggers, etc ;-)
Of course, I play a bit Devil’s Advocate here, and I’m one of those people forced to know every reason why various features got removed, but I somehow feel that lots of actual improvements (like protocol buffers) could be done without doing the stripping. Also, I know that most of the features removed are not harmful in any way, if not used :)
In the end, most of heavyweight database work is done at storage engine layer anyway, most of resource usage is by storage engine, most of scalability troubles are at the storage engine, and most of actual needed improvements and features should be done at the storage engine layer.
domas mituzas 
Hi!
I think you raise a number of interesting points here, especially the problem of creating portable, independent modules that at the same time can work together.
The pluggable storage engine interface is a point in case: how can we really use a module with separate features if the SQL dialect that we use to command the module does not change at the same time? It is clear that the current situation in MySQL does not address this: some SQL is engine specific and thus ignored by other engines, and at the same time, some engines have more features than can be addressed through the available SQL constructs. I guess you could argue that your ‘pot of spaghetti’ would solve this, but it seems to me that would still kind of take the shine of the concept of ‘pluggable’ storage engines. I mean, a third party would still need to either make do with all available SQL syntax, or persuade the server developers to add yet more engine specific syntax.
Now I am not saying this to complain or whatever, I am just interested in what people suggest can be done against this situation, and how realistic it is to repair this. I mean, I can imagine that there somehow exists an extensible parser, and that the engine can plug in some new syntax to command it. But how realistic is this? Does anybody know of a system that supports such flexibility for plugins? Another possibility would be to add some clause to statements that are meant to pass through clauses that are interpreted directly by the engine, but then this would still mean that each engine is more than just a datastore, it would still need to ship its own parser to deal with all that.
/me is confused how these problems can be solved.
regards,
Roland.