2004/06/10

2004 06 10

Base | Lab | MCSE | Servers

21:34

Disabled DHCP

Neither of these machines needs to be a DHCP server, since from their point of view, there already is one on the network.

In fact, the DHCP service is provided by the VMWare networking subsystem.

I removed the DHCP role from both boxes.

Base | Lab | MCSE | Servers

21:31

Oh, what a glorious sight...

Two virtual machines, running simultaneously, all boote dup in under five minutes. Pity they don't fit side by side on the host desktop, but it still is a magnificent sight; I can't help but thinking about the enourmous amount of complex software that has to work together just for these two windows to do what they are doing right now. (i.e. nothing much)

The NIC icons on both of the machines are constantly flashing; I can't wait to have them up and running like this and to slap a network sniffer inbetween them.

Base | Lab | MCSE | Servers

21:29

Small tweaks

Individually, the servers will perform less, but together they should be prevented from pushing the host overboard. So I tweaked the settings a little bit:

  • RAM was decreased to 196MB per machine. Why that number? Because I like it, that's why. :)
  • Both servers are now wired to use the same virtual CD-ROM drive. I assume those drives are configured to be read-only anway, so that should not be a problem in itself.
  • I synchronised the configurations: I disabled all the fancier features of VMWare, like snapshotting and drag-n-drop between the host and the guest.
  • Inside both guests, I decreased the desktop resolution to 800x600. Looks like crap, but I need to save on resources.

Base | Lab | MCSE | Servers

21:20

Two on one

After days of sslloowwllyy installing servers, I am now in a position to have them running side by side on the host machine. Fingers crossed...

Base | Lab | MCSE | Servers | Server02

16:47

OS Installation

With the newly created bootable ISO a a virtual CD-ROM drive, I kicked off the installation of this second server. Sure enough, things went by quicker than the installation of the first one, but that might just as well be because of not using the external hard drive anymore.

I busied myself with reading about hotfixes and slipstreaming and every now and then entering a few details for the installation process.

I used the same options as for Server01, other than (obviously) the server name.

On several occasions, I got the impression that all activity inside the virtual machine had stopped, only to 'wake up' again when I switched the focus back to the window. Also, I switched back on various occasions when the installation was waiting at a prompt. Not in the least, I was also downloading some fairly large files and playing music through iTunes. All in all, I think these were not ideal circumstances for a fast installation. Nevertheless, it took only a little over an hour and a half to complete, including the installation of VMWare tools. (every reboot takes around 10 minutes, so time does go by quickly)

Base | Lab | ISOs

16:45

I created a bootable ISO

This post is really only a placeholder for the result: I managed to create a bootable ISO file from the Windows Server 2003 installation disk, but it has been nothing short of an ordeal to get to this point. I will try to record my adventures at a later time, using the notes below (which you can ignore for the time being).

Nero Express will only take a boot image from a floppy drive, ISO buster can extract everything and then some, but can't create bootable images, mkisofs should be able to, but consistenly fails to respond to what I believe to be a corect command line, MagicISO would probably work painlessly if it would not be crippled to less than 300MB. Undisker did the trick, even though it claimed errors at the beginning of the process.

Base | Lab | MCSE | Servers | VMWare

11:02

Using an ISO image instead of a CD drive

Whilst I was on the VMWare site looking for the exact command line parameters to use in a custom shortcut for Server01, I came across the marvellous idea to create an ISO file from a CD-ROM and mount that ISO file as a virtual disk in a virtual machine.

That would allow me to do all kinds of install without having to even have the Windwos Server 2003 CD available, or even the drive.

What's more, the transfer of files at installation time is likely to go a lot faster than when reading from an external device.

Base | Lab | MCSE | Servers | Server01

10:50

Setting up the paging file

The situation is a bit weird in this case, but I think the basic idea still holds: putting the OS files and the Virtual Memory Paging File (commonly known as a swap file) on the same partition is asking for trouble, if only because of fragmentation.

At the very least, I always try to create a seperate partition to use for swap, if not an actual seperate disk. The maximum wisdom in these matters is to divide your swap requirement over as much bus capacity as possible, but avoiding otherwise busy areas. So: not on the same partition as the OS. If there are two physical disks, then one-third on the disk that also has the OS and the rest on the other disk. Never on two partitions on the same disk; that would just make the head move around like mad and slow things down all around.

In this case, everything actually resides on the same file system on the same disk in the Host OS, so the logic hardly applies. My only reasons for creating a seperate virtual disk and using that is that the fragmentation issue is still relevant and that it is more manageable, in case I decide to expand the disk setup in future.

Like real ones, virtual disks can be swapped in and out of a system or replaced with bigger or smaller ones, but doing the same with partitions requires much more difficult procedures and low-level fiddling with tools like PartitionMagic. All in all, I would just a much avoid having to that altogether.

In the end, I formatted the (very much overcapacitated) swap drive and moved the swap file from the OS drive onto the swap drive.

Base | Lab | MCSE | Servers | Server01

10:45

Disks moved to the host's hard drive

As outlined earlier, I have moved the virtual disks to the hard drive of the host machine, my laptop. The assumption was that that would make a difference in performance. Well ... let's just say that my instincts were right: the Virtual machine with Server 2003 installed has been tuned down to 'merely' 384MB of RAM and it still starts up three times faster than it did when its disks were physically on the external drive.

Base | Lab | iTunes

10:29

How to preserve library data when moving music files

Updated June 2007!

I think iTunes is great as a player, because it allows me to build all kinds of so-called 'Smart' playlists, using some easy rules that can be combined to wonderfully complex things, all just in order to come up with the right song to play next.

Problem is, this being Apple software, it has a tendency to give you the choice of:

  1. letting it control your life (shut up and be happy) or
  2. giving you absolutely nothing to work with if you decide to opt out of preconfigured bliss.

Specifically: if you allow iTunes to 'organise your music folder' (meaning: totally butcher any ideas that you might have yourself about what makes up a logical way of organising your music files) you are allowed to move the library (i.e. the folder(s) that hold the music files) around to another place on your drive, to other disks or even to a network drive.

The reverse is the actual problem: move a music file and iTunes will treat it as a new file, which obviously resets play counts and 'Last Played' dates.

Here are a few things I tried and found NOT to work:

  • Edit the 'iTunes Music Library.xml' file with your favourite text editor only to find it ignored first and overwritten second.
  • Delete the binary 'iTunes Music Library.itl' file and your entire library will vanish.
  • Import a previously exported or a modified XML file and the entries will be treated as new files.

I searched a lot of fora, but all relevant posts and discussions I could find are either OS X oriented, using AppleScript that (AFAIK) does not run on Windows or advising you to take the bliss option outlined above. I did read something about AppleScripts possibly working on Windows, but it would be confined to within iTunes itself, meaning that moving stuff around in the Windows file system is out of reach. Running AppleScripts that try to do things with the Finder (the OS X interface to the file system) are likely to do nothing if you're lucky and serious damage if you are not.

Admittedly by means of sheer luck on a what-does-this-button-do-mission I have stumbled on to a working solution by myself! Hurrah!

The trick is very simple: do not delete the 'itl' file, but rather damage it. To be sure, just empty it. Upon startup, iTunes will try to load its database and fail (because you have damaged it). In an effort to repair it, it will (re)import the XML file--including our precious play statistics. A message follows that the itl file has unfortunately become corrupt and all that could be done has been tried. It even saves a copy of your emptied file. :)

Given that the XML stores not only music data, but also the playlist definitions, you are good to go from this point on.

Please see below for a number of gotchas and observed side-effects.

Instructions for technically timid folk (on Windows!):

  1. Shut down iTunes
  2. Make a backup copy of these files:
    • iTunes Music Library.xml (on my system, it is located in My Documents/My Music/iTunes)
    • iTunes Music Library.itl (on my system, it is located in My Documents/My Music/iTunes)
    • If you have the disk space to spare and you are truly paranoid, backup your actual music files as well.
  3. Open the itl file in Notepad (or any other text editor, but not in Word or any other word processor).
  4. Select all (Ctrl+A) and hit Delete. You should now have an empty file; save it and close the text editor.
  5. In Explorer, verify that the file size is indeed now 0Kb. I haven't tried partially damaging the file, but this method is a sure-fire way.
  6. If you haven't already, do whatever you want to do to your music folder(s): move them all to a new drive, shuffle some of them around, etc.
  7. Open the xml file in your favourite text editor and modify the paths as they should be, reflecting the new location of your music files. Global search and replace comes to mind as a quick way of making these changes. Don't worry about mistakes--you did take a backup, did you?
  8. Save the file and fire up iTunes. Instead of starting normally, it will import the xml file. I have some 6500-odd entries in it and a dozen playlists. With the files on an external USB2.0 drive, it takes a few minutes for this to complete, which is indicated by the warning message about how iTunes discovered that your library file had gotten damaged and how it tried to save the day. Click OK.
  9. If all is well, you should now be looking at your library, with all data intact and the locations of the music files updated.

Some additional notes:

The location of files is dependant on your system defaults and any specific choices you may have made during installation of iTunes. In any case, you should be aware of the following:

  • The iTunes database files (the .itl and .xml files) are installed in your system's 'My Music' folder. Typically, this is a subfolder of 'My Documents'. Note that this location can be different for different users of the same computer. I do not know what happens in a situation where there is a 'Shared Documents' type of setup.
  • The music files (typically, your mp3 files) can be anywhere on your system. If you have let iTunes 'manage' your music library, it will have put (a copy of) those files in subfolders of the top-level library folder.
    This is where the confusion sets in: the default configuration is for the top-level library folder to be the same folder in which the database files reside. You CAN (easily) change the location of the library folder (in the preferences) but you CAN NOT change the location of the database files. At least, not from within iTunes and not without mucking about with profile defaults in Windows itself.
  • Changing the top-level library folder through the preferences only tell iTunes where to expect existing files and where to put new ones. That is, if you have configured it to manage your files for you. If you have not told it to manage your files, changing the preference does (I think) nothing for the music files.
  • Since the Album Art functionality was introduced, there is another tree of subfolders underneath the database folder, containting the image files for your albums. I honestly don't know how the location of those files changes (or not) if you change preferences. Just be aware that they are there, for the time being.
  • Moving the whole folder around (someone moved their entire 'My Music' folder to an external drive) confuses the hell out of iTunes. The actual program files (the software, rather than the data) are stored elsewhere (on Windows, 'C:\Program Files' by default). When starting iTunes, it can tell (from the Registry on Windows, in some similar way on OS X) where the database files are supposed to be located. The database files will point the way to the music files.
    At best, iTunes might be blunt and recreate the database files (empty!) at the place from where you have just moved them. Other weird side effects may occur. Bottom line: do NOT move the database files, but ONLY the music files themselves.

Some more notes and comments, partially inspired by emails I have received over the years:

  • Beware of some text editors that are unicode aware or have any other reason to stick (normally) invisible characters in the first few bytes of your files. People have reported crashes using my method and at least one person mentioned that UltraEdit had done just this. Removing the extra garbage from the text file using another editor solved the crashing.
  • If you have a large collection, NotePad is likely not up to the task. If you find that you can not open your XML with NotePad, try a text editor of heavier caliber, such as UltraEdit or LargeEdit. I personally use vim, which I suppose is akin to long range missiles in terms of caliber. :)
  • I have the setting enabled to normalise the volume of all tracks. Upon re-importing the xml, iTunes will 'Determine song volume' on all the songs. Depending on the size of your library, this might take a while, but it is otherwise harmless. Just let it do its thing.
  • Since iTunes started handling album art work, the setting for 'automatically downloading missing art work' might trigger a similar automatic process, taking another good bit of time to complete.
  • The same applies to the option for 'determining gapless playback'.
  • My method works on both Windows and OS X.
  • I have had confirmation of the method working on just about every version of iTunes up until the latest (at the time of writing this, June 2007). I have reason to believe that it will remain working for a while longer, because of the following: the xml file exist for a purpose, which is to allow Spotlight and other applications on the OS X platform access to iTunes' data. This is a cornerstone of cross-application functionality in OS X, on which a good many third party programs depend. I speculate that the existence of the XML file is pointless on Windows, but since iTunes is developed for the two platforms simultaneously, it has been overall easier for the development team to just leave it in. (rather than having to maintain two different architectures). Also, it seems to me that the 'repair' facility depends on the xml file to be present. My best guess is that as long as the cross-application API on OS X continues to use the XML mechanism, my method will keep working.

  • I have briefly looked into the way playlists are stored in the XML file. At first glance, it seems to be in the format of actually listing the IDs of tracks that belong to a particular playlist, along with copies of the metadata. This holds true even for dynamic playlists. When you change around some of the meta-data (such as the location of the file) these lists seem to get upset. I have had reports of playlists only partially surviving the process described above.
  • Podcast subscriptions are reported to break. I don't use them myself, so I can not give you any more specifics than that.
  • Someone (named Greg) mentioned that the 'Date added' data element will be reset, since the repairing is seen as re-importing. I have not verified this and actually have my doubts about it.
  • Someone (named Paul) moved an external drive from one machine to the next (I gather a new installation) and after installation dropped in his 'old' database files. Since all the paths to the music files were kept identical between installations, it worked straight away.
  • Be sure to exit out of iTunes cleanly at least once, so that it will rebuild the itl and xml files to be in sync again. I can't say what will happen if you don't, but based on past experience, I would say that only Bad Things can happen from that.
  • I have received feedback from a handful of happy users whose playlists have been saved from forced re-arrangement. If you use the tip above, do me a favour and send me a message at schmolle@pobox.com. Oh, and mention iTunes or SchmolleWorld in the subject, so that you don't get caught in my spamfilter straight away. :)
  • I can't claim having done a proper test, but the timing of the feedback suggests that the above should work on at least the last version or two of iTunes. Specifics in your feedback are welcome.
  • From one or two fora I can deduce that this trick will work on Mac OS X. There is no itl file, though. Rather, an iTunes Library and an iTunes Music Library.xml file. You need to have you Finder set to exposing file extensions to see this, obviously.
  • In a recent version, Apple introduced this method. All good and well--if you let iTunes organise your library. So bummer about that one.