More image similarity analysis... & my last resort method, rules for the eyeball

I found some useful image analysis code in Matlab online from siddhant ahuja. I found it during an handy google search on "texture similarity image analysis" or some such.  I will be using some of the transforms to see if I get any good results...?

Similarity Matrix for MFCC's of the audio Variations V from Chicago Cage 100 Festival* :

Similarity Matrix for movement from Variations V from Chicago Cage 100 Festival* (from video analysis, tracking torso and wrists by hand, then using the summed velocity)

Similarity Matrix for movement during the ballet Giselle (a sort of null hypothesis), movement information obtained in the same way as the previous example.*

* These are excerpts, not from the entire piece.

If all else fails, very quickly I will have to go to less computational means of analysis. Create strict rules for what similarity means, and create a ranking. I can then move on to use this as a justification for further work in my conclusion. This is very dissatisfying but this has been an enormous amount of brute force, technical, and programming work for what should be a music history paper for a class, so at some point I have to stop the search for the holy grail self-similarity matrix similarity...

Literature Search & Analysis with Self-Similarity Matrix Tutorial, Part Deux

Lit Search

Useful Matlab tools for audio feature analysis:
http://www.audiolabs-erlangen.de/meinard/data/

I am reading his textbook on MIR, the chapter on musical structure to see if it has any aha's for me. Alas, not so far. Although it has made me think about extracting different features...

I am now extracting volume and MFCC's for my audio clips... which I think make sense. But I might also extract STMSPs, since that might be better than just volume. I'm not sure, due to the nature of Variations V.

Foote's papers are a good place to start for applications of similarity matrices to music & video
http://www.rotorbrain.com/foote/vanity.htm

I thought this one could be helpful in comparing similarity matrices but I'm not sure:
http://pdf.aminer.org/000/009/514/summarizing_video_using_non_negative_similarity_matrix_factorization.pdf

btw -- Non-Negative Matrix Factorization Toolbox for Matlab
https://sites.google.com/site/nmftool/

OK, I have found more helpful papers on the technical side of self-similarity maps, but still no actual examples of any actual musical analysis applications in actual musicology/music theory papers. I have one article on Interlibrary Loan which MAY have something I can use... but if that lead dies, then I'm considering this part of the lit search over, and at least, have the examples that may sorta apply. I have an interesting performance analysis of some of Xenakis' works so that should help guide writing style and presentation, I guess... Le sigh.

I think I'm not done with some of my analysis... it really is an art, sometimes, statistics.

Octave codin' or, OK, so where was I in all this analysis??

Construct a self-similarity matrix in Octave/Matlab

octave:143> beats = wavread('/Users/me/mywave.wav'); % read in the files
octave:143> beats_dist = pdist(beats); % pairwise distance btw all the points
octave:144> beat_self_matrix = squareform(beats_dist); % turns it a square matrix
octave:144> imshow(beat_self_matrix); % show the matrix visually


Those are the basics but then, you actually need to downsample, smooth the data, & actually extract a feature (MFCC, amplitude, STMSP, CENs)

In terms of the movement data, I am dealing with 2d x,y data that I collapse in distances and then velocity measures. Since this data is from 2D video analysis I also smooth the data via an averaging filter and then divide by the standard deviation.

Here's a folder of the octave functions I've written for self-similarity maps. The audio versions may have more general use, but the movement ones do assume that you've collapsed your features into one vector.  They are documented somewhat sketchily, so use with care. They require the signal and statistics packages from Octave, and the packages that THOSE require. Well, ya know.