Monday, November 04, 2013

Combining lots of ".aiff" files using sox

There was a memorial service yesterday, and the sound guy at the church made an audio CD. I'm surprised everything fit, really; it was about 81 minutes.

My dad noted that the CD had 81 tracks, each about a minute long. When trying to listen to the CD on my mom's computer, he could only hear a minute at a time, and a track would typically end in the middle of a song or sentence. Then he'd have to hit the "Play" button and wait for the CD to spin up again.

I supposed that there was no actual pause on the CD, but that the CD player software just stopped when it saw a track marker. To make this friendlier for playback on computer, we decided to concatenate the "tracks" into a single long file. We could divide it up into smaller pieces (inserting a track marker when a particular song or speech starts, say) later.

My guess is that the sound guy had a single very long audio track, and he told his CD burning software to insert a track marker every 60 seconds. Why was this a good idea? Well, suppose you're playing the CD on a CD player (these things still exist). Some CD players don't tell you how far into a given track they've gotten. So if you wanted to find a spot 23 minutes into the thing, you'd be pretty much stuck with either waiting 23 minutes after you hit "PLAY," or using the skip/review function (if your CD player has one) while listening... assuming of course that you knew what came before and after the point of interest.

With the every-60-second track markers, you can skip 23 tracks. It still might start in the middle of a song or sentence, but that's better than waiting 23 minutes, right?

My newphew came into the room, and upon hearing the problem, proceeded to copy 81 "AIFF" files into a directory on the computer's hard drive. (Do audio CDs actually have AIFF files? No, but on Mac OS X that's what it looks like in the Finder.) This took a long time. After looking at the Activity Monitor app and seeing no CPU or RAM hogs, we decided that having a 98% full hard drive might have something to do with the issue. That took care of some of our waiting time, anyway.

Anyway, he seemed to remember hearing that Quick Time 7 could concatenate sound files, and he tried it, taking the first file (I think the filename was "1 Track 1.aiff" or something like this) and doing the click-and-drag thing with "2 Track 2.aiff". After some seconds of computing, the computer showed us that we indeed had a two-minute file. We played it and there was no audible gap or pause as we crossed the one-minute mark.

But nobody had the patience to drag the other 79 files, and I certainly didn't feel confident that we could do it without error. Attention wanders when doing tasks like that. The worst thing was, if we did make a mistake (leaving out track #24 for example, or appending track #72 twice), we might not notice it for some hours.

So we did a web search on concatenating .AIFF files. One result mentioned using cat(1), so I gamely opened a terminal window. My thought was to get all the files listed in order, then pass them to cat, which would concatenate them end-to-end. We would route the output to an output file, say x.aiff.

How to get a list of all the files? Well, one could type something like "echo *.aiff" and get a result that looks something like this:

% echo *.aiff
1 Track 1.aiff 10 Track 10.aiff 11 Track 11.aiff 12 Track 12.aiff
13 Track 13.aiff 14 Track 14.aiff 15 Track 15.aiff 16 Track 16.aiff
17 Track 17.aiff 18 Track 18.aiff 19 Track 19.aiff 2 Track 2.aiff
20 Track 20.aiff 21 Track 21.aiff 22 Track 22.aiff…
Actually the result is one very long line, which ends as follows:
79 Track 79.aiff 8 Track 8.aiff 80 Track 80.aiff 81 Track 81.aiff 9 Track 9.aiff
Is the first problem obvious? When we say "echo *.aiff" or whatever, the output is sorted character-by-character, so for example "19" comes before "2" because the first character of "19" (i.e., "1") sorts lower than the first character of "2". The ls command does that too. If you knew the files were written in a certain order (e.g., track 2 has an earlier modification time than track 19 for example) then ls might help, but since the source was an audio CD with no mtime data, this didn't occur to me.

What did occur to me, though, was to say "sort -n"; as the manpage tells us (type "man sort")

   -n     Compare according to arithmetic value an initial numeric  string
          consisting of optional white space, an optional - sign, and zero
          or more digits, optionally followed by a decimal point and  zero
          or more digits.
So if we said "ls|sort -n"...
collin@v2:~/tmp/CD-tracks 
% ls *.aiff|sort -n
1 Track 1.aiff
2 Track 2.aiff
3 Track 3.aiff
4 Track 4.aiff
5 Track 5.aiff
6 Track 6.aiff
7 Track 7.aiff
8 Track 8.aiff
9 Track 9.aiff
10 Track 10.aiff
…
78 Track 78.aiff
79 Track 79.aiff
80 Track 80.aiff
81 Track 81.aiff
collin@v2:~/tmp/CD-tracks 
% 
OK, that's more like it. Suppose then we piped all these to cat(1). Umm, let's try with just the first three files.
collin@v2:~/tmp/CD-tracks 
% ls *.aiff | sort -n | head -n3
1 Track 1.aiff
2 Track 2.aiff
3 Track 3.aiff
collin@v2:~/tmp/CD-tracks 
% 
The head command gives the first <howevermany> lines of its input. To say how many lines (the default I think is 10), we use -n# where # is the number of lines we want -- in this case, three.

Given the three lines, how do we pass all of them to cat? Easiest thing is the xargs command. If it had been invented last week, there would probably be a patent application in the works, but fortunately the idea came in an earlier era so all can use it for free. Here's the thing: If you give xargs five lines, then it will append them to the end of a single command line. So if we said for example

collin@v2:~/tmp/CD-tracks 
% ls *.aiff | head -n3 | xargs echo
1 Track 1.aiff 10 Track 10.aiff 11 Track 11.aiff
collin@v2:~/tmp/CD-tracks 
% 
Right? We can do that with cat and send the output to a file we'll call "x", maybe like this:
collin@v2:~/tmp/CD-tracks 
% ls *.aiff | sort -n | head -n3 | xargs cat > x.aiff
cat: 1: No such file or directory
cat: Track: No such file or directory
cat: 1.aiff: No such file or directory
cat: 2: No such file or directory
cat: Track: No such file or directory
cat: 2.aiff: No such file or directory
cat: 3: No such file or directory
cat: Track: No such file or directory
cat: 3.aiff: No such file or directory
collin@v2:~/tmp/CD-tracks 
% 
Were you surprised, or did you wonder how cat would be able to tell where one filename ended and the next began? Let's get rid of the spaces in those filenames. There's any number of ways to do that, but I typed something like this to see if it would work:
collin@v2:~/tmp/CD-tracks 
% for X in *.aiff; do echo mv "$X" ${X// /.}; done | head -n3
mv 1 Track 1.aiff 1.Track.1.aiff
mv 10 Track 10.aiff 10.Track.10.aiff
mv 11 Track 11.aiff 11.Track.11.aiff
collin@v2:~/tmp/CD-tracks 
% 
What's "${X// /.}"? Well, let me go back to the beginning of the line.
  1. for X in *.aiff; do
    says to assign the variable X to successive values of whatever filenames match *.aiff, and execute everything until the done keyword.
  2. echo &hellip
    I want to see what's about to happen, rather than just executing it.
  3. mv "$X" …
    The command mv is the Unix™ command "move", which is how we rename things. What do we rename? The $X says "whatever's in shell variable X".

    Why do I put the $X in "double quotes"? Because that tells the shell to treat the entire name (1 Track 1.aiff for example) as a single "word".

    What new name will we give to $X? That's coming next.

  4. ${X// /.}
    This tells the shell to start with the variable X (which will be "1 Track 1.aiff" the first time through, etc.) and replace all instances of " " by ".". How did I know this? From reading the manpage for the shell. Type "man sh" or "man bash" some evening when you're having trouble getting to sleep:
    BASH(1)                                                          BASH(1)
    
    NAME
           bash - GNU Bourne-Again SHell
    
    SYNOPSIS
           bash [options] [file]
    …
    Parameter Expansion
       The `$' character introduces parameter expansion, command substitution,
       or arithmetic expansion.  The parameter name or symbol to  be  expanded
       may  be enclosed in braces, which are optional but serve to protect the
       variable to be expanded from characters immediately following it  which
       could be interpreted as part of the name.
    …
       ${parameter/pattern/string}
       ${parameter//pattern/string}
          The pattern is expanded to produce a pattern just as in pathname
          expansion.  Parameter is expanded and the longest match of  pat-
          tern  against  its  value is replaced with string.  In the first
          form, only the first match is replaced.  The second form  causes
          all  matches  of pattern to be replaced with string.  If pattern
          begins with #, it must match at the beginning  of  the  expanded
          value  of parameter.  If pattern begins with %, it must match at
          the end of the expanded value of parameter.  If string is  null,
          matches  of  pattern are deleted and the / following pattern may
          be omitted.  If parameter is @ or *, the substitution  operation
          is  applied to each positional parameter in turn, and the expan-
          sion is the resultant list.  If parameter is an  array  variable
          subscripted  with  @ or *, the substitution operation is applied
          to each member of the array in turn, and the  expansion  is  the
          resultant list.  
  5. done;
    see item 1 above.
Convinced that that ought to work now, I do this:
collin@v2:~/tmp/CD-tracks 
% for X in *.aiff; do mv "$X" ${X// /.}; done
collin@v2:~/tmp/CD-tracks 
% ls *.aiff | sort -n | head -n3 | xargs echo cat
cat 1.Track.1.aiff 2.Track.2.aiff 3.Track.3.aiff
collin@v2:~/tmp/CD-tracks 
% 
That's more like it. Now let's do the real thing:
collin@v2:~/tmp/CD-tracks 
% ls *.aiff | sort -n | xargs cat > x.aiff
collin@v2:~/tmp/CD-tracks 
% 
Great! We looked at the size of each file (about 10 Mbytes per track, except the last), and the output file, x.aiff, was about 800 Mbytes, so we were OK. Opening the result in Quick Time Player 7, we found it was only as long as the last track, i.e., less than 40 seconds.

Disillusionment

Well, we went back to the web search, to the post that mentioned using cat, and found that basically, you can't do that.

I did another web search and found that sox can in fact concatenate sound files, including aiff files. Yes! We went to the sourceforge site and downloaded the Mac OS X zip file. I think it was a zip file anyway (actually happened about 12 hours ago). Unlike a lot of Mac OS apps where you click here to install, etc., this one you just unpack and run it. I ended up typing something like

% ~/Downloads/sox-14.2/sox… 
but I'm getting ahead of myself.

First we looked at the documentation for sox, which told us that if you want to concatenate sound files, you put them on the command line, with the output file last. If I recall correctly the example was

% sox short.aiff long.aiff longer.aiff
but of course we had 81 input files. What would the output file be? I would want it to sort last, so I did something like this:
% touch 999.aiff; ls *.aiff | sort -n | xargs ~/Downloads/sox-14.2/sox
Then, opening 999.aiff in Quick Time Player 7, we found that it indeed appeared to be about 81 minutes long.

Technology is great when it works. I copied the result to my USB flash drive, so the lovely Carol can listen to it after I return home.

No comments: