Comments on: INSERT speed, paramy, auto-inc http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/ Sat, 17 Dec 2011 20:14:59 +0000 hourly 1 http://wordpress.com/ By: paramy - import dumps in a flash | All things Sysadmin http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1283 Fri, 30 May 2008 21:34:43 +0000 http://dammit.lt/?p=118#comment-1283 [...] Follwing an interesting and lengthy discussion one evening on whether multi threaded data importing should be part of the client or the server, Domas went ahead and scratched his itch to write something and came up with paramy. [...]

]]> By: Domas Mituzas http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1282 Wed, 28 May 2008 14:01:59 +0000 http://dammit.lt/?p=118#comment-1282 4 core server, 8 sessions. pthread was nowhere near a bottleneck.

]]> By: Mark Callaghan http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1281 Wed, 28 May 2008 13:18:37 +0000 http://dammit.lt/?p=118#comment-1281 How many CPU cores and how many concurrent sessions did you use? pthread functions should use the most time on a 4 or 8 core server with many sessions.

]]> By: Domas Mituzas http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1280 Wed, 28 May 2008 08:20:14 +0000 http://dammit.lt/?p=118#comment-1280 Mark,

My profiles showed a lot of work done in checksums (calculating both old and new checksum). I’d gladly have entirely disabled checksums during initial data load – and enable calculation of them later on.

Do you still have your COMPACT patch? };-) I’m not sure we should care about 20% win for our largest tables, as they sit quite cold anyway.

]]> By: Mark Callaghan http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1279 Tue, 27 May 2008 22:56:31 +0000 http://dammit.lt/?p=118#comment-1279 By waiting for 5.0.64, you fortunately have avoided the pain that ruined a weekend for me. I haven’t tried to compare MySQL4 with MySQL5 since then. There is more overhead in InnoDB because of the COMPACT row format, and even if you don’t use it, you pay the CPU cost. The best way to see that cost is to run without concurrent inserts so that you don’t get any overhead from SMP issues in InnoDB. On the bright side, the COMPACT row format reduced my database size by 20%.

I spent another weekend removing support for the COMPACT row format from InnoDB, but I decided to not submit the patch. I was able to reduce the CPU overhead.

]]> By: pabloj http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1278 Tue, 27 May 2008 10:53:00 +0000 http://dammit.lt/?p=118#comment-1278 Looks slow from my tests too, and it’s v5.1.23 in my case.

]]> By: Domas Mituzas http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1277 Tue, 27 May 2008 05:49:36 +0000 http://dammit.lt/?p=118#comment-1277 Mark, this was 5.0.64 – and profiling was showing mostly innodb internals. I’ll experiment a bit more with profiles.

]]> By: Mark Callaghan http://dom.as/2008/05/26/insert-speed-paramy-auto-inc/#comment-1276 Mon, 26 May 2008 22:39:34 +0000 http://dammit.lt/?p=118#comment-1276 5.0 uses much more CPU for insert statements because of the parser. At least half of the regression was fixed in 5.0.54.

]]>