Sounds like you’re wanting to use midi files as a score for your visuals? I haven’t used midi myself very much (mostly due to its quantization / numerical limitations, but these might not matter to you for visual cues or whatever you’re doing), but I bet you can find some simple code around to parse the files. A quick search brought up this library, though I haven’t looked too close at it. From there, it shouldn’t be too difficult to sequence those events back from your update loop.
I believe there are both Windows and OS X specific ways of reading midi files, though I can’t find them at the moment and I’m not sure if there is a way to use them without creating sound. As in, just for reading / sequencing the file. Might be.
YSE looks really interesting @paul.houx! And open source, I bet there’s lots of good stuff in there.