keepcalculator

06 Numbers and Powers

Fix MySQL Failing to Start After a Database Crash

Card No. 06·39707 · Updated · Standard formula, shown below

If MySQL refuses to start after a crash, the cause is usually a corrupted InnoDB table, a missing or damaged redo log, or a leftover lock file. This guide walks you through a safe, step-by-step recovery: first you’ll identify the error from the logs, then you’ll force InnoDB recovery to get the server running, back up your data, and finally repair or restore the affected tables. You’ll need command-line access to the MySQL data directory and the ability to stop/start the service (or run mysqld manually).

What You’ll Need

  • Root or sudo access to the server (or an account that can manage the MySQL service).
  • MySQL’s data directory path (usually /var/lib/mysql on Linux, C:\ProgramData\MySQL\MySQL Server 8.0\Data on Windows).
  • Ability to edit MySQL configuration files (my.cnf or my.ini).
  • Enough free disk space for a full backup of the data directory (or at least the database folders).
  • MySQL client tools (mysql, mysqldump) – these come with the server installation.

Step-by-Step

  1. Stop MySQL completely. If it’s running, stop it cleanly: sudo systemctl stop mysql (or service mysql stop, or stop the Windows service). Do not kill the process with kill -9 unless it’s hung – a clean stop avoids further corruption.
  2. Check the MySQL error log. The log is usually at /var/log/mysql/error.log (Linux) or in the data directory (Windows). Look for the last few lines – they’ll tell you why it’s failing. Common messages: “InnoDB: Corrupted page”, “InnoDB: Database page corruption on disk”, “InnoDB: Unable to lock ./ibdata1”, or “Can’t open the mysql.plugin table”. Write down the exact error – it guides the next steps.
  3. Try a simple restart. Sometimes the crash left a stale lock file. If the log shows “Unable to lock …” or “Address already in use”, remove the stale lock file: sudo rm /var/lib/mysql/*.pid (or *.lock on some systems). Then try starting MySQL again. If it starts, you’re done – but still check table integrity (step 9).
  4. Enable InnoDB recovery mode. If the log shows corruption, edit your config file (my.cnf / my.ini) and add under the [mysqld] section: innodb_force_recovery = 1. Start MySQL. If it starts, great – the crash was minor. If it still fails, increase the value to 2, then 3, and so on, up to 6. Each level skips more checks and makes the server less safe, so use the lowest value that works.
  5. Back up everything. Once MySQL starts (even in recovery mode), immediately back up all databases with mysqldump --all-databases --single-transaction --routines --triggers > backup.sql (if InnoDB) or mysqldump --all-databases --skip-lock-tables > backup.sql (if MyISAM). If the server won’t start even with recovery=6, copy the entire data directory to another location as a raw backup – you’ll need it later.
  6. Repair InnoDB tables. With recovery mode still on, connect to MySQL: mysql -u root -p. For each database, run CHECK TABLE dbname.tablename; to see which tables are corrupt. Then repair them with REPAIR TABLE dbname.tablename; (this works for MyISAM and some InnoDB cases). If REPAIR fails, you can try OPTIMIZE TABLE or, for a specific table, export the data with SELECT ... INTO OUTFILE and re-import.
  7. Remove recovery mode. After repairing or confirming no corruption, set innodb_force_recovery = 0 (or remove the line) in the config. Restart MySQL normally. If it starts cleanly, run a full CHECK TABLE on all tables to be safe.
  8. Restore from backup if needed. If you can’t repair the tables, restore the backup you made in step 5. Stop MySQL, move the corrupted data directory aside, and restore the backup (either the SQL dump or the raw file copy). Then start MySQL.
  9. Verify data integrity. After the server is up, run mysqlcheck -u root -p --all-databases to check for any remaining issues. Also test a few key queries on your most important tables to ensure data is readable.

Common Mistakes

  • Setting innodb_force_recovery too high too quickly – this can cause more damage by skipping crucial checks. Always start at 1 and increment only if needed.
  • Forgetting to remove the recovery setting before restarting normally – leaving it at 4+ can prevent writes and cause silent data loss.
  • Running REPAIR TABLE on InnoDB tables while the server is in recovery mode – it can fail or worsen corruption. It’s safer to use mysqldump to export and re-import.
  • Ignoring the error log – guessing the cause wastes time. The log almost always points to the exact problem.
  • Not backing up before attempting repairs – if a repair goes wrong, you have no fallback.

Tips

  • Keep a recent mysqldump backup off-site – it’s the simplest insurance against crashes.
  • Enable the innodb_force_recovery setting only temporarily – never leave it on in production.
  • If you have a replica (slave) server, promote it to primary instead of spending hours recovering the crashed one – then fix the crashed server later.
  • Use the innodb_buffer_pool_dump_at_shutdown=ON and innodb_buffer_pool_load_at_startup=ON settings to speed up recovery after a crash.
  • Consider using a filesystem that supports snapshots (like LVM or ZFS) – you can roll back to a pre-crash state in seconds.

Troubleshooting

If the steps above don’t get MySQL running, try these:

  • Still won’t start with recovery=6? The corruption is severe. Restore from your raw backup (step 5) or from the last known good mysqldump. If you have no backup, you can try using the innodb_force_recovery=6 to start MySQL in read-only mode and then use SELECT ... INTO OUTFILE to extract data from surviving tables.
  • MySQL starts but tables are empty? The redo log (ib_logfile*) may be corrupted. With recovery mode on, you can delete the redo logs (only if you have a backup!) – MySQL will recreate them. Be careful: this can cause data loss if transactions weren’t flushed.
  • Error says “Table ‘mysql.plugin’ doesn’t exist”? The system tables are corrupted. Restore the mysql database from a backup, or reinitialize the data directory (which wipes all databases – last resort).
  • Can’t connect to MySQL after starting? Check the error log for socket or port issues. Ensure MySQL is listening on the right port and that the socket file exists.
  • Repair table hangs or crashes the server? Stop the repair, increase innodb_force_recovery by one level, and try again. If it still fails, export the table’s data using SELECT ... INTO OUTFILE and re-import into a fresh table.

Frequently Asked Questions

Will I lose data if I use innodb_force_recovery?

Possibly. Recovery mode skips some checks and may ignore uncommitted transactions. Always back up your data before using it, and use the lowest setting that works.

Can I repair a corrupted InnoDB table without losing data?

Sometimes yes, with REPAIR TABLE or by exporting and re-importing the table. But if the corruption is severe, you may lose some rows. That’s why backups are essential.

What does the error “InnoDB: Database page corruption on disk” mean?

It means a page (typically 16KB) of data in an InnoDB table is unreadable or has an invalid checksum. This is often due to a hardware issue, power loss, or a bug. InnoDB recovery can sometimes skip the bad page, but you’ll need to repair or restore the affected table.

How long does MySQL recovery take?

It depends on the size of the database and the severity of corruption. A small database might recover in seconds, while a large one with many corrupted pages can take hours. Recovery mode with high force levels may be faster but less safe.

Can I prevent MySQL crashes from corrupting my database?

Use a reliable power supply, enable innodb_flush_log_at_trx_commit=1 (default) for durability, and regularly back up. Also consider using a RAID array or a filesystem with checksums (like ZFS) to detect hardware issues early.

What if I don’t have a backup of the data directory?

You can still try to start MySQL with recovery mode and export what you can. If the system tables are intact, you can use mysqldump. If not, you may need to extract data from the raw .ibd files using specialized tools, but that’s complex and not always successful.

Related

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.

More cards in this drawer

The whole Numbers and Powers drawer →