Tag: dba

MySQL DBA, python edition

In the age of jetsetting and space travel and ORMs and such, MySQL DBAs are the least sophisticated ones nowadays, usually fighting terabytes or petabytes of data with army of shell scripts – as there’re no nice frameworks to explain what you want to do in MySQL administration. The nice thing about proper object frameworks is that they allow to concentrate on the work and logic done, allowing to think on the process done, rather on languages/APIs/etc.

For example, moving a slave to another master down a replication topology could be expressed this way (this is a working code, actually):

slave = mysql(options.slave)
oldmaster = mysql(slave.get_master())
newmaster = mysql(options.newmaster)

oldmaster.lock()
oldpos = oldmaster.pos()
newmaster.wait(oldpos)
newmaster.lock()
oldmaster.unlock()
slave.wait(oldpos)
slave.change_master(newmaster)
newmaster.unlock()

I’m sure transaction group/global IDs would simplify the process a lot, but still, having building blocks one can write pretty much self-documenting narrow code, shuffle actions done without having to rethink whole programming logic too much. Implementation of methods like .sync(), .clone(), .promote() ends up environment-specific, but may save quite some time afterwards too.

As much as I’d like everyone around to get their data management actions written down into scripts, I’d like every DBA action I do to be written down in such code too :-) I’d love to have code, which detects resource shortages, orders servers, deploys software and re-shards data automatically… well, you know what I mean :)

DBAs of all countries, unite!

I’m observing the process of most awesome SHOW commands being abolished, destroyed and some weird information_schema tables are introduced instead.

Say, even though you can select configuration variables using @@syntax, you can’t do same for much more interesting to DBAs status variables in any more interesting logic.

Apparently instead of doing

SHOW STATUS LIKE "questions"

one has to do this now (I’m being dramatic here, above hasn’t been removed yet, but hasn’t been expanded for better usage either):

SELECT VARIABLE_NAME, VARIABLE_VALUE
FROM INFORMATION_SCHEMA.GLOBAL_STATUS
WHERE VARIABLE_NAME="QUESTIONS"

Do note, those SQL standard followers will get caps-lock button swapped with space bar soon.

Of course, we, DBAs, know that one can simplify stuff by creating stored routines:

CREATE FUNCTION `gstatus`(v varchar(64)) returns varchar(1024)
return
( SELECT variable_value
  FROM information_schema.global_status
  where variable_name=v LIMIT 1
)

So we can do such simple things as:

mysql> select m.gstatus("questions");
+------------------------+
| m.gstatus("questions") |
+------------------------+
| 140                    |
+------------------------+
1 row in set (0.00 sec)

Of course, this leads to solution of one of most common DBA problems, how to get decent status variable values per time:

CREATE PROCEDURE m.report(in timer float)
begin

DROP TEMPORARY TABLE IF EXISTS status_old;
CREATE TEMPORARY TABLE status_old
SELECT * FROM INFORMATION_SCHEMA.GLOBAL_STATUS;

SELECT SLEEP(timer) into @x;
SELECT
    s.variable_name status,
    (s.variable_value-o.variable_value)/timer value
FROM INFORMATION_SCHEMA.GLOBAL_STATUS s
    JOIN status_old o USING (variable_name)
WHERE s.variable_value>0;

DROP TEMPORARY TABLE status_old;

end

So, the “show me changes-per-second for values in last 0.5s” would look like this:

ysql> call m.report(0.5) //
+-----------------------------------+---------+
| status                            | value   |
+-----------------------------------+---------+
| ABORTED_CLIENTS                   |       0 |
| ABORTED_CONNECTS                  |       0 |
| BYTES_RECEIVED                    |  532662 |
| BYTES_SENT                        | 1140894 |
...
| QUERIES                           |    2884 |
| QUESTIONS                         |    2878 |
| SELECT_FULL_JOIN                  |       2 |
| SELECT_RANGE                      |     196 |
| SELECT_SCAN                       |     146 |
...
| THREADS_CACHED                    |      12 |
| THREADS_CONNECTED                 |     -28 |
| THREADS_CREATED                   |       4 |
| THREADS_RUNNING                   |      -2 |
| UPTIME                            |       2 |
| UPTIME_SINCE_FLUSH_STATUS         |       2 |
+-----------------------------------+---------+
125 rows in set (0.53 sec)

Query OK, 0 rows affected, 1 warning (0.54 sec)

So, by spending five minutes on writing very simple INFORMATION_SCHEMA procedure we can resolve one of usual nightmares in MySQL DBA environments.

I can get back now to the initial idea of this post – if one DBA can write such small neat thing in few minutes, would you imagine how useful can a collaboratively built repository of DBA-assisting stored procedures in functions, and how we can spit at all the SQL standard verbosity, and make our systems easy to manage? :) I think we shouldn’t allow not to share such utilities, as widespread use and “expect it already there” would make overall work much much easier. Let’s use and reuse (and someone should set up framework for building such thing ;-))