{"id":189,"date":"2016-05-01T00:00:04","date_gmt":"2016-05-01T07:00:04","guid":{"rendered":"http:\/\/4gi.wtf\/wp\/?p=189"},"modified":"2021-12-08T10:17:24","modified_gmt":"2021-12-08T18:17:24","slug":"making-linux-bootable-clones","status":"publish","type":"post","link":"https:\/\/looseassociations.com\/?p=189","title":{"rendered":"Making Linux Bootable Clones"},"content":{"rendered":"<p>A bootable clone is a disk that holds a complete copy of a running system that\u2019s ready to boot. Bootable clones can be a critical part of your backup strategy.<\/p>\n<p><!--more--><\/p>\n<h1>Why Bootable Clones<\/h1>\n<p>If you already know why you want a bootable clone, just <a href=\"#TheRecipe\">skip to the recipe<\/a>.<\/p>\n<p>If you\u2019re truly hardcore, and just want a command reference so you don\u2019t leave out any steps, skip to the <a href=\"#ExpertMode\">expert mode recipe<\/a>.<\/p>\n<p>If you have mission critical computers in your life, I hope you are already aware of the importance of good backups. I\u2019ve noticed, though, that a lot of people think that just having copies of all their files is enough. They\u2019ll set up, say, an automated cloud backup \u2014 and figure they\u2019re done.<\/p>\n<p>They\u2019re probably in for a nasty surprise if disaster hits.<\/p>\n<p>A lot of the information in your computer isn\u2019t in the files, but in various nebulous kinds of metadata: where the files are located, various configuration files, license keys, special directories, file links, permissions\u2026 Just restoring all your files isn\u2019t enough if you want to be back online as fast as possible.<\/p>\n<p>My primary laptop is a macOS machine. I\u2019ve long employed <a href=\"http:\/\/www.shirt-pocket.com\/SuperDuper\/SuperDuperDescription.html\" target=\"_blank\" rel=\"noopener noreferrer\">SuperDuper<\/a> and <a href=\"https:\/\/bombich.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Carbon Copy Cloner<\/a> to make bootable clones. If my laptop gets destroyed, lost, or stolen, all I have to do is plug a clone into any recent-vintage Macintosh, reboot, and I\u2019m back in production. I don\u2019t even have to wait for files to copy. By contrast, last year I tried a scratch restore from a Time Machine backup, and it took almost two days to get reasonable working system, and several weeks after that before most of the little glitches were smoothed out. In fairness, my setup is a lot more complicated than the average user, but I also get the sense that most users are largely unaware of how tied they are to complicated configurations that they\u2019ve (consciously or otherwise) fine-tuned to their workflow over the course of years.<\/p>\n<p>I also maintain a hundred or so Linux-based systems. I\u2019ve searched from time to time, but surprisingly I\u2019ve never found a SuperDuper-like utility for making bootable clones. It\u2019s long been on my list of things to create, but a recent <a href=\"https:\/\/risley.net\/?p=521\" target=\"_blank\" rel=\"noopener noreferrer\"><code>systemd<\/code>-related disaster<\/a> left me with a crashed mission-critical server for over five days. That, combined with a horrible bug in the <code>duplicity<\/code> backup system, resulted in more downtime than all of my systems combined over the previous ten years.<\/p>\n<h1>Current Options<\/h1>\n<ul>\n<li><a href=\"http:\/\/clonezilla.org\/\" target=\"_blank\" rel=\"noopener noreferrer\">Clonezilla<\/a> isn\u2019t a bad option. It will create a sector-by-sector clone onto a new hard drive. It is file-system agnostic so your copy is almost guaranteed to work as well as the original. Optionally, you can put your clones into image files. They won\u2019t boot, but can be restored to then-bootable media. However:\n<ul>\n<li>Clonezilla requires that you boot from a live CD to make your backups. That means that your system is unusable during the hours that it can take to make the backup (not acceptable for mission-critical servers).<\/li>\n<li><del>Clonezilla requires that the system have a working optical drive<\/del>.<\/li>\n<li>When using Clonezilla, you\u2019re pretty much stuck with working from the console (or spending time configuring <code>ssh<\/code>). In my world, that means a certain amount of physical discomfort. I much prefer the comfort of working from my mother\u2019s basement, chugging Mountain Dew while my Dorito-dust covered fingers clack away on my sticky gaming keyboard.<\/li>\n<li>Clonezilla requires that the destination disk be as large or larger than the source. I once tried to make a clone of a 2 terabyte server disk to an \u201cidentical\u201d 2 TB disk. For whatever reason (I didn\u2019t investigate, possibly some defective sectors), the destination drive was a few blocks smaller than the source disk. After many hours of copying, the backup failed.<\/li>\n<\/ul>\n<\/li>\n<li>Filesystem copies using <code>cp<\/code>, <code>tar<\/code>, or <code>rsync<\/code> can make sure that you get all your files. The usual advice is then \u201cinstall a fresh system, and copy the important files from your backup to the new bootable system.\u201d Guess what: it won\u2019t work. Yes, you can copy your <code>\/home<\/code> directory, but that won\u2019t restore any of your system configuration. Over time, you\u2019ve probably installed multiple software packages with multiple configuration files, perl and python modules, all scattered through the <code>\/etc<\/code> and <code>\/var<\/code> directories and configuration files, thoroughly mixed in with install-specific files and data that you can\u2019t just replace with information from the old system. You\u2019ll lose log files. You\u2019ll lose startup sequences. Unless you run a vanilla system (and I doubt any plain-vanilla users are here reading this) you\u2019ll be in for hours and hours of reconfiguration and figuring out which \u201cimportant\u201d files should be copied, and which must not be, and which configuration files need to be carefully merged with those on the new system.<\/li>\n<li><code>dd<\/code>. I use and love <code>dd<\/code>, and it meets my criterion for simplicity, but it has some real limitations.\n<ul>\n<li>The destination drive has to be as large or larger than the source drive.<\/li>\n<li>It isn\u2019t smart about what it copies. I have a server with a 1.5TB drive, but it\u2019s only using 86GB (only a fraction of which changes between backups). When using <code>dd<\/code>, I pretty much have to copy the entire 1.5TB which takes hours. Yes, I could re-partition the drive and only <code>dd<\/code> the partition but then the clone wouldn\u2019t be bootable and I\u2019d lose the simplicity.<\/li>\n<li>It doesn\u2019t do incremental backup.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h1>Simplicity is Key<\/h1>\n<p>People get cagey about backups. They want to be selective (who needs to be able to access three years of temporary cache files?). They want to conserve backup space. That\u2019s all well and good, but your bootable clone is not the place to do this. Just buy a backup drive that\u2019s as large or larger than your primary and back up <em>everything<\/em>. You don\u2019t want to be scrambling to get back online and discover that a rule you created to save a few MB of drive space inadvertently blocked the backup of a critical file.<\/p>\n<p><a name=\"TheRecipe\"><\/a><\/p>\n<h1>Creating a Bootable Clone<\/h1>\n<p>This technique has been tested with Ubuntu 14.04, Ubuntu 18.04, Devuan ascii, as well as recent Arch releases. It should work with Debian-derived distros and other systems that use the <code>GRUB2<\/code> bootloader.<\/p>\n<h2>Get Ready<\/h2>\n<ul>\n<li>plug in backup drive<\/li>\n<li>open a <code>bash<\/code> shell; you need to be root so\u2026<\/li>\n<\/ul>\n<p><code>sudo -s<\/code><\/p>\n<ul>\n<li>enter your passphrase<\/li>\n<li>next, figure out which drive you\u2019re working with<\/li>\n<\/ul>\n<p><code>fdisk -l<\/code><\/p>\n<ul>\n<li>and identify which drive is the backup drive; we\u2019re going to assume its <code>\/dev\/sdx<\/code>. So remember: whenever you see <code>\/dev\/sdx<\/code>, be sure to replace the x with the appropriate device indicator.<\/li>\n<\/ul>\n<h2>First Time? Prepare the Bare Drive<\/h2>\n<p><strong>WARNING!<\/strong> This will erase all data on whatever drive you specify. Specify the wrong drive, and you\u2019ll erase your <em>source<\/em> instead of your destination. Be careful!<\/p>\n<ul>\n<li>If you already have a bootable clone, <a href=\"#SkipDriveInit\">skip the next three steps<\/a> and you\u2019ll just incrementally update your clone.<\/li>\n<li>The next three steps initialize a bare drive for first-time cloning. Whatever device you specify <b>*will be erased,*<\/b> so make sure you\u2019re erasing the disk you think you are. Remember to change <code>\/dev\/sdx<\/code> to your actual device designation. You might have to repeat the \u201cd\u201d command several times if there are multiple existing partitions. When in doubt, use defaults.<\/li>\n<\/ul>\n<p><code>fdisk \/dev\/sdx<\/code><\/p>\n<ul>\n<li>here\u2019s what your fdisk session should look like \u2014 though you will have to display the list of partition types and use the version-specific code for the EFI and swap partitions:<\/li>\n<\/ul>\n<pre>Welcome to fdisk (util-linux 2.29.2).\r\nChanges will remain in memory only, until you decide to write them.\r\nBe careful before using the write command.\r\n\r\nCommand (m for help): <em><strong>d<\/strong><\/em>\r\nPartition number (1-3, default 3):\r\n\r\nPartition 3 has been deleted.\r\n<em><strong>[repeat until you get an error because all the partitions have been deleted]<\/strong><\/em>\r\n\r\nCommand (m for help): <em><strong>n<\/strong><\/em>\r\nPartition type\r\np primary (0 primary, 0 extended, 4 free)\r\ne extended (container for logical partitions)\r\nSelect (default p):\r\n\r\nUsing default response p.\r\nPartition number (1-4, default 1):\r\nFirst sector (2048-468862127, default 2048):\r\nLast sector, +sectors or +size{K,M,G,T,P} (2048-468862127, default 468862127): <em><strong>+512M<\/strong><\/em>\r\n\r\nCreated a new partition 1 of type 'Linux' and of size 512 MiB.\r\n\r\nCommand (m for help): <em><strong>t<\/strong><\/em>\r\nSelected partition 1\r\nPartition type (type L to list all types): <em><strong>XX<\/strong><\/em>\r\nChanged type of partition 'Linux' to 'EFI (FAT-12\/16\/32)'.\r\n\r\nCommand (m for help): <em><strong>n<\/strong><\/em>\r\nPartition type\r\np primary (1 primary, 0 extended, 3 free)\r\ne extended (container for logical partitions)\r\nSelect (default p):\r\n\r\nUsing default response p.\r\nPartition number (2-4, default 2):\r\nFirst sector (1050624-468862127, default 1050624):\r\nLast sector, +sectors or +size{K,M,G,T,P} (1050624-468862127, default 468862127): <em><strong>+213G\r\n[subtract the desired swap size from the total disk size; this was a 223GB disk as reported by fdisk, and I want about a 10GB swap]<\/strong><\/em>\r\n\r\nCreated a new partition 2 of type 'Linux' and of size 213 GiB.\r\n\r\nCommand (m for help): <em><strong>n<\/strong><\/em>\r\nPartition type\r\np primary (2 primary, 0 extended, 2 free)\r\ne extended (container for logical partitions)\r\nSelect (default p):\r\n\r\nUsing default response p.\r\nPartition number (3,4, default 3):\r\nFirst sector (447744000-468862127, default 447744000):\r\nLast sector, +sectors or +size{K,M,G,T,P} (447744000-468862127, default 468862127):\r\n\r\nCreated a new partition 3 of type 'Linux' and of size 10.1 GiB.\r\n\r\nCommand (m for help): <em><strong>t<\/strong><\/em>\r\nPartition number (1-3, default 3):\r\nPartition type (type L to list all types): <em><strong>XX<\/strong><\/em>\r\n\r\nChanged type of partition 'Linux' to 'Linux swap'.\r\n\r\nCommand (m for help): w\r\nThe partition table has been altered.\r\nCalling ioctl() to re-read partition table.\r\nSyncing disks.\r\n<\/pre>\n<ul>\n<li>Format the UEFI partition. You might need to install dosfstools (eg. <code>apt install dosfstools<\/code>) to create the FAT partition.<\/li>\n<\/ul>\n<pre>mkfs.fat -F32 \/dev\/sdx1<\/pre>\n<ul>\n<li>Create the file system. Remember to edit <code>\/dev\/sdx2<\/code> appropriately.<\/li>\n<\/ul>\n<p><code>mkfs.ext4 \/dev\/sdx2<\/code><\/p>\n<ul>\n<li>Set up the swap partition.<\/li>\n<\/ul>\n<pre>mkswap \/dev\/sdx3<\/pre>\n<ul>\n<li>Mount the file system. Remember to edit <code>\/dev\/sdx2<\/code> first.<\/li>\n<\/ul>\n<p><code>mount -t ext4 \/dev\/sdx2 \/mnt<\/code><\/p>\n<ul>\n<li>Create some special directories.<\/li>\n<\/ul>\n<p><code>mkdir \/mnt\/dev<br \/>\nmkdir \/mnt\/dev\/pts<br \/>\nmkdir \/mnt\/sys<br \/>\nmkdir \/mnt\/proc<\/code><\/p>\n<p><a name=\"SkipDriveInit\"><\/a><\/p>\n<h3>Mount and Copy<\/h3>\n<ul>\n<li>Mount the clone you\u2019re updating. Skip this command if you\u2019re starting from scratch, as you\u2019ve already mounted the new drive.<\/li>\n<\/ul>\n<p><code>mount -t ext4 \/dev\/sdx2 \/mnt<\/code><\/p>\n<ul>\n<li>Because we\u2019re using <code>rsync<\/code>, if a clone already exists on this drive, it will incrementally update which should save some time.<\/li>\n<\/ul>\n<p><code>rsync --archive --verbose --delete --exclude=\/dev --exclude=\/sys --exclude=\/proc --exclude=\/mnt -xx \/ \/mnt<\/code><\/p>\n<h3>Install GRUB2<\/h3>\n<ul>\n<li>Now you have the files copied, but you have to install a bootloader to make the drive bootable. To do this, we\u2019ll chroot to the clone disk after making some special system directories available there.<\/li>\n<\/ul>\n<p><code>mount --bind \/dev \/mnt\/dev<br \/>\nmount --bind \/dev\/pts \/mnt\/dev\/pts<br \/>\nmount --bind \/sys \/mnt\/sys<br \/>\nmount --bind \/proc \/mnt\/proc<br \/>\nmkdir \/mnt\/boot\/efi<br \/>\nmount \/dev\/sdx1 \/mnt\/boot\/efi<br \/>\nchroot \/mnt<\/code><\/p>\n<ul>\n<li>Install the bootloader. Remember that you have to fix up <code>\/dev\/sdx<\/code> <em>in two places<\/em> before executing these commands.<\/li>\n<\/ul>\n<pre>grub-install \/dev\/sdx\r\ngrub-install --recheck \/dev\/sdx\r\nupdate-grub<\/pre>\n<h3>Clean up <code>\/etc\/fstab<\/code><\/h3>\n<ul>\n<li>This is a trickier part of the process. Systems are likely to be configured to use other disk partitions for <code>\/boot<\/code> or <code>\/home<\/code> or for swap space. In order to keep things clean and simple, I\u2019ve made the clone into a monolithic file system. Unfortunately, the cloned <code>\/etc\/fstab<\/code> might cause the system to fail to boot. These commands attempt to create an <code>fstab<\/code> that\u2019s good enough to boot. (Swap might be important to you; see the note at the end of this article for information on re-enabling swap on a clone.)<\/li>\n<li>First, back up the existing <code>fstab<\/code>.<\/li>\n<\/ul>\n<p><code>mv \/etc\/fstab \/etc\/fstab.bak<\/code><\/p>\n<ul>\n<li>In this command, look out for the buried <code>\/dev\/sdx2<\/code> that needs to be edited!<\/li>\n<\/ul>\n<p><code>echo -e \"UUID=`lsblk -no UUID \/dev\/sdx1`\\t\/\\text4\\tdefaults,noatime\\t0\\t1\" &gt;\/etc\/fstab<\/code><\/p>\n<h3>Clean Up and Reboot<\/h3>\n<ul>\n<li>Your clone is complete, but lets clean up a bit.<\/li>\n<\/ul>\n<p><code>exit<br \/>\numount \/mnt\/boot\/efi \/mnt\/dev\/pts \/mnt\/dev \/mnt\/sys \/mnt\/proc \/mnt<\/code><\/p>\n<ul>\n<li>When you want to test your clone (and you do!), you\u2019ll have to abandon the comfort, safety, and copy\/paste convenience of your remote shell and work from the console, as you have to modify boot parameters<\/li>\n<li>During the boot process, press <code>del<\/code> or <code>esc<\/code> or <code>F11<\/code> or <code>F12<\/code> or whatever key allows you to change the BIOS or EFI boot device, and change it to the newly created disk<\/li>\n<\/ul>\n<p><a name=\"ExpertMode\"><\/a><\/p>\n<h1>Expert Mode<\/h1>\n<ul>\n<li>Here\u2019s the whole cloning recipe without the commentary. Remember that it won\u2019t work (and can be dangerous!) without editing, but hardcore folks can put this in a window next to their terminal for copypaste profit.<\/li>\n<\/ul>\n<pre>sudo -s\r\nfdisk -l\r\nfdisk \/dev\/sdx ### fixup\r\nmkfs.fat -F32 \/dev\/sdx1 ### fixup\r\nmkfs.ext4 \/dev\/sdx1 ### fixup\r\nmkswap \/dev\/sdx3 ### fixup\r\nmount -t ext4 \/dev\/sdx2 \/mnt ### fixup\r\nmkdir \/mnt\/dev \r\nmkdir \/mnt\/dev\/pts \r\nmkdir \/mnt\/sys \r\nmkdir \/mnt\/proc\r\nrsync --archive --verbose --delete --exclude=\/dev --exclude=\/sys --exclude=\/proc --exclude=\/mnt -xx \/ \/mnt\r\nmount --bind \/dev \/mnt\/dev \r\nmount --bind \/dev\/pts \/mnt\/dev\/pts \r\nmount --bind \/sys \/mnt\/sys \r\nmount --bind \/proc \/mnt\/proc \r\nchroot \/mnt\r\nmkdir \/boot\/efi\r\nmount \/dev\/sdx1 \/boot\/efi\/ ### fixup\r\ngrub-install \/dev\/sdx ### fixup\r\ngrub-install --recheck \/dev\/sdx ### fixup\r\nupdate-grub \r\nmv \/etc\/fstab \/etc\/fstab.bak\r\necho -e \"UUID=`lsblk -no UUID \/dev\/sdx1`\\t\/\\text4\\tdefaults,noatime\\t0\\t1\" &gt;\/etc\/fstab ### fixup required\r\nexit \r\numount \/mnt\/boot\/efi \/mnt\/dev\/pts \/mnt\/dev \/mnt\/sys \/mnt\/proc \/mnt<\/pre>\n<h1><a name=\"After_Words\"><\/a>After Words<\/h1>\n<p>Remember that the \u201cdisaster\u201d in \u201cdisaster recovery\u201d can mean many things, from acts of nature to government seizures to theft or sabotage or facility breakdown. I keep two bootable clones of each machine and, each week, I swap them between two geographically-separate sites. That way, I always have a bootable clone that\u2019s less than two weeks old even if one location is completely lost. Of course, that\u2019s in addition to much-more-frequent <code>rsync<\/code> backups between sites and archiving cloud backups. Back up in depth \u2014 it\u2019s good for your health.<\/p>\n<p>No, I don\u2019t actually juggle 200+ clones. Many of my boxen are very small (think whiteboxed routers). I use a small stack of 2TB portable USB3 drives and create <code>rsync<\/code> images in separate directories under root. If I need a clone, I can either copy the subdirectory to root on a fresh drive, or just move the subdirectory up to root. I then install <code>GRUB2<\/code> and go.<\/p>\n<p>Unlike the Apple use case above, using clones to run from different hardware isn\u2019t quite as straightforward. Since the hardware doesn\u2019t come from a corporate monoculture, more differences exist than in the Macintosh ecosystem. Surprisingly, this hasn\u2019t been as big an issue as I expected, in spite of the fact that my infrastructure is built from fire-sale, castoff, and various other forms of junk computers ranging from Mac minis to repurposed doorstops. It is sometimes necessary to tweak <code>\/etc\/fstab<\/code> or to change networking parameters to switch from <code>eth0<\/code> to <code>eth1<\/code> or the like.<\/p>\n<p>In order to keep things as simple as possible, I have not enabled swap (though we did reserve a swap partition). Linux systems tend to degrade more gracefully when swap is enabled. The ArchWiki has a good discussion on <a href=\"https:\/\/wiki.archlinux.org\/index.php\/swap\" target=\"_blank\" rel=\"noopener noreferrer\">how to enable swap<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A bootable clone is a disk that holds a complete copy of a running system that\u2019s ready to boot. Bootable clones can be a critical part of your backup strategy.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-189","post","type-post","status-publish","format-standard","hentry","category-coding"],"_links":{"self":[{"href":"https:\/\/looseassociations.com\/index.php?rest_route=\/wp\/v2\/posts\/189"}],"collection":[{"href":"https:\/\/looseassociations.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/looseassociations.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/looseassociations.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/looseassociations.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=189"}],"version-history":[{"count":14,"href":"https:\/\/looseassociations.com\/index.php?rest_route=\/wp\/v2\/posts\/189\/revisions"}],"predecessor-version":[{"id":539,"href":"https:\/\/looseassociations.com\/index.php?rest_route=\/wp\/v2\/posts\/189\/revisions\/539"}],"wp:attachment":[{"href":"https:\/\/looseassociations.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=189"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/looseassociations.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=189"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/looseassociations.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=189"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}