t
7
e
s |
Tom 7 Entertainment System Hero (show and demo reel)
(16 Apr 2009 at 20:14) |
Hey, okay. Finally I have the videos ready to reveal secret project 7H, which is called Tom 7 Entertainment System Hero.
This is part video game and part performance art piece. The video game is essentially an implementation of Guitar Hero, where the songs are Tom 7 Entertainment System tunes. Some of these are ridiculously intricate and most have weird time signatures, which makes for advanced play. It supports keyboard on Mac, Windows, and linux or real USB guitar controllers (like the XBox 360 ones) on Mac and Windows. That includes accelerometers and whammy bar. The best introduction is to watch the demo reel:
As usual, for best results click through to the HD version, or download the 1080p AVI from that page.
This video has a bunch of clips in it, mostly from the Show at Belvedere's. You'll see a bunch of things. One is that I actually mess up a lot when playing. I'm better than this but two things contributed to my mistakes: (1) I was kinda drunk since the show started like 3.5 hours late and I got free beer for being a "musician" and (2) in the last week before the show I was sprinting to get all the software and hardware working, so I actually didn't practice hardly any of these songs more than the one time it took me to decide to put them in the setlist. Once it's available publicly I will challenge you to high-score battle to prove it. Hardware you ask? I didn't build the guitar or drums of course, but I did build the Laser Suspension Womb, which to be more pithy I sometimes call my "USB laser hat." It's a hardhat with a bunch of very bright LEDs and actual laser diodes embedded in it, powered by 1.5A, worn on the head, and connected to the computer via USB implemented on a custom circuit board with a PIC microcontroller. The in-game music and events ("drums were kidnapped!") trigger the lights and lasers to play along. I have a clever hack so that it doesn't need special drivers on any platform, though that's not helping penetration much because there's only one of them. This was my first real hardware project in my adult life, but now that I know how to do it I hope to do more (especially input devices, i.e., "instruments"). It's much better in 2009 than I recall from sticking paperclips and resistors in the parallel port in 1993.
Rock Band drums are supported too. Unlike the guitar, which has a goal pattern for you to match, these are totally freeform. Commodore 64 samples are played in response to drumhits. I wish the controller supported some kind of velocity sensitivity, because that is kind of important for drum expressiveness, but too bad so sad.
Techno details: The implementation is almost all in Standard ML using SDL, except for the low-level sound synthesis thread and the interface to the USB laser hat. That stuff's in C. It's easy to mix them. The code has some shortcuts in it for sure and deserves to be cleaned up (lots of them introduced in that last week sprint) but it also has some really nice parts, like the algorithm that matches your input to the score. The matching is ambiguous, so there's an on-line dynamic programming algorithm to be maximally generous to your playing. (I don't think Guitar Hero II had this maximally generous algorithm, which was one of the reasons I started working it out like a year and a half ago, but I do think that GH III and on do it right.) The finger patterns you're supposed to match with the guitar, which I call the "score", is generated automatically from MIDI files. To turn a T7ES MIDI file into a T7ES Hero game file, I have to assign instruments to each of the tracks, and then pick which tracks or track parts are supposed to be played on the guitar. The rest is automatic, save some tuning parameters. "genscore" has a model for how closely a candidate score matches the original MIDI (for example, if consecutive MIDI notes are rising in pitch, then it's better for the fingers to also be rising on the fretboard) and then it solves for the optimal assignment, measure-by-measure. I thought that I would need to modify the score after this to get good quality, but it actually works amazingly well. Some of the stuff it comes up with is super fun, like I would assume was created by a human with a good sense of fun. No. Only cyber-brain.
Here's the last two songs of the regular set in full. In this you see that you can actually play drums and guitar at the same time if you're good enough (I am not). 2 player mode? Maybe soon:
I don't know if I'll ever get the opportunity to perform this again (befriend T7ES Band Page on Facebook for guaranteed notification), but maybe. Either way, I'm looking forward to sharing the software with you guys, which I will do as soon as I finish the auto-update and high score table, so that we can compete with each other and I can release song packs. |
 |
|
| Sweetness. Nice architectural diagram. I want to hear more about the ambiguity of input/score matching. |
Oops... sorry.
Also I was gonna say you didn't even mention the random band name generator which I found by browsing the code. "Facial Fish" for life! |
This makes me wish I had usb guitar hero stuff for serious.
Really, really cool. |
| Excellent project! And finally I know what a laser suspension womb is! :) |
I keep on wondering where you get the time to do this kind of stuff....
not that I can see yet, as the Vimeo thingie says the clips are temporarely offline. |
i'm with Arthur -- vimeo says the videos are unavailable.
What a clever idea! :) |
The video on vimeo is temporarily unavailable!! No!!!! Anyway before you wrote this post I secretly espied the Belvedere's video, and I have to say your GH style is quite awkward. Therefore, once I actually get off my ass and get around to buying an Xbox and Rock Band you should expect some serious challenges, as I was quite the GH2 connoissieur back in my day (as well as the current connoissieur of some other choice rhythm games).
The auto midi-to-score sounds sweet too. Does it support hammer-ons and the like? It's a little hard to tell from the video and I can't watch it again anyway.
I'm interested in what the ambiguous matching algorithm is like too. Usually more "serious" rhythm games (like those wacky Bemani ones) would be like X ms and deal with it, but they also have different levels of matching as opposed to yes/no. Doesn't that make it hard to compare scores fairly?
Also, this might just get me to finish my electronic drums-to-Rock Band interface project, which has been languishing for quite a time. |
| Actually it looks like none of the videos on vimeo are available right now, for me at least. Hooray!!! |
| Oh I remembered what else I was going to say: it needs bonus levels and WELCOME TO WARP ZONE. |
NICE!!
Too bad that unlike Escape, T7ESH probably won't be allowed in Fedora.
http://www.redhat.com/archives/fedora-games-list/2008-January/msg00022.html |
The ambiguity in matching comes from hammer ons and pull-offs. When you finger a button, it could be because you are hammering on that note (if you are currently in-streak) or it could be because you're preparing to play it with a strum. Similarly for pull-offs. It's noticeable if you don't have support for this and use an eager algorithm or one based solely on time. There's a little bit of subtlety about whether you're on-streak, too.
Yes, the auto score generation does hammer-ons and pull-offs automatically too, though it's just based on their proximity to the previous note, which works pretty well it turns out.
It wouldn't be that hard to extend it to score based on how temporally close you were to the matched note, but I don't like that level of persnicketiness, because I feel like if I'm rocking out I should be given some leeway for "swing".
Adam: That is too bad. Anyway good luck getting USB to work on linux. ;)
Adhesion: In the show I played, right before the last segment, it literally goes "WELCOME TO WARP ZONE". No kidding. What do you mean by my style being awkward? It's a bit unconventional I guess, in that I always poke for maximum accuracy. I can't think of anything else weird about it, except maybe how high I strap the guitar (but that's true with real guitars even). |
Vids working again, and I gotta say...
OMG? |
| It's 'cause you're all downstroke Tom! |
| Go Standard ML! |
| i'm so proud to see your baby finally working |
| Awesome! It's come a long way since you showed me. The drums especially seem SUPER fun. And I want to find out what kind of LEDs (or laser diodes?!) you used in the laser suspension womb - I keep losing those fscking bike lights, who wants to pay $15 for an LED in some cheap plastic!? |
Thanks guys! The LEDs are these: http://www.sparkfun.com/commerce/product_info.php?products_id=8860
They are crazy bright. If you got 10 of them you could definitely beat the bike lamps bigtime, but they are 80mA each so you're going to need a lot of juice. The laser diodes are:
http://www.sparkfun.com/commerce/product_info.php?products_id=594
They are great lasers but didn't come out as well on the hat as I thought they would. |
Now I have a macbook pro and I found your Mac OS X build... but the question is: how do I get that running?
(in the meantime: considering buying a guitarhero controller for this) |
| I'm guessing you mean the OSX_build sir in subversion? That is just the SDL libraries for Mac. It does make the Mac the easiest platform to compile for, but it's still a bit of an ordeal because you need to install gcc and mlton first. You could try, but I recommend waiting for the binaries unless you like that kind of stuff (I don't!) |
Beauteous.
This is certainly the best version of "live electronic music" I have ever seen... |
| maybe a smoke machine in an inflammable backpack would enhance the portable ambiance of the laser suspension womb... |
| You mean flameproof or you mean easy to ignite? |
Perhaps a future direction in which to take the Laser Suspension Womb:
http://www.youtube.com/watch?v=Opf5jIukSBM |
| Damn, that sounds great! I want one! |
Well, if you just poke, you can't do alt-strumming. I guess there's not much tremolo picking in chiptune music, though, so it's not a big loss.
Forgive me if you've talked about this in older posts, but when you write stuff like Many Happy Returns, do you actually think up riffs in 40/16 time in the first place or do you decide the meter first and then shove stuff into it? The former would be pretty impressive. |
I strum when it's necessary, of course, but it's much less precise. There are some songs with note rushes (rt2i comes to mind) that need it in T7EShero. I can't even imagine an electric toothbrush poking fast enough. I don't remember if any of the songs I played at Belvedere's were strummy.
Also, there are achievements ("medals") for playing a song with all poking or all plucking in T7EShero, so don't you want extra credit??
Hard to answer the second question. It's usually not the case that I shove into a grid unless I am setting myself to some challenge (Theme from Goog). I usually let the song unfold in a linear way and then adjust the time signature to meet its natural periodicity. But I wouldn't say that I "think up riffs" in that I hum them or something and then put them to piano roll; I really compose in the editor, thinking about the next few notes of the sequence and playing the bits as it goes. In the specific case of Many Happy Returns it was a brush its hair approach for sure. The beginning two bars in "51/16 time" were placed sort of stream-of-consciousnessly, then I made the same tune more rock 'n' roll for the later 42/16 part. (And again, these aren't really measures of 42 beats, it's really some amalgam of shorter measures with changing signatures, sometimes different for different parts, but the periodicity is 42 beats.) |
| what is the final song in the demo reel? (the one in 'advance mode', no tongues please.) |
| That's "Rutgers". One of my new-school faves. |
| awesome, thanks, i kept having it stuck in my head after watching the video. great song.. |
| Rutgers is rulez. |
| stalin forever |
|
|
|