p
e
r
s
o
n
a
l |
The adventures of learnfun and playfun, part 2
(23 Aug 2013 at 18:15) |
A few days ago I posted a follow-up video to the one I made for my SIGBOVIK paper this April. It covers 6 new games, and I already have the recordings for another 7, coming soon. This new video is a bit more polished in production values but along the same lines, new surprises notwithstanding. :) If you haven't seen the first video you should watch that one first (now over 500,000 views!) since it explains the basics.
I also put up downloadable binaries on the learnfun & playfun site so if you are patient enough to use the command line and follow instructions (and have 64-bit windows and lots of RAM), but not patient enough to figure out how to compile the damn thing yourself, now's your chance.
Three hours from now starts Ludum Dare #27, which I'm participating in as usual! I'm psyched to create something, but I have no idea what, yet... |
 |
|
| You seem to be missing a closing double quote for a HTML attribute in this post, and that makes part of the text unreadable to me. Could you please fix it? |
| oops, I guess I am still an HTML noob! fixed, thanks! :) |
I tried to run the learnfun.exe but it kept spitting out CHECK FAILED (!moviename.empty). But i put it in exactly how you said to put it in. I even checked the learnfun.cc file. I put the config.txt & ninja.fm2 & ninjagaiden.nes file inside of the highest playfun folder. my config.txt reads
game ninjagaiden
movie ninja.fm2 |
| Is there a newline after the "movie" line? Someone reported that is required. |
No.... I better try that....
(damn it Tom put it in the readme :(.) |
| Got it. It works! |
| Sorry... expect some rough edges; it's just research software! :) |
Your research is very inspiring. :)
I read in the paper that playfun looks approximately 300-500 frames into the future in order to decide the best scenario to choose from. Did I get that right?
Anyway I'm very curious to know just how far do you think you can extend this, because the end result should be making the program "smarter" in the long run?
300 frames should be roughly 5 seconds, but if that were extended to perhaps 20 seconds (if that's possible) it should mean the program makes better choices, correct?
Thanks and sorry if I'm a noob at this! I'm just very interested in your program's capabilities. :) |
| It could be extended very far; if I had 10x the number of computers (or time) then 50 seconds. This wouldn't make it make better choices, though... the problem is that there are a lot more than 10x the number of 50 second input sequences than 5 second sequences (exponentially many!) so the likelihood of happening upon one that is good drops dramatically. I chose 5 seconds because it seemed reasonably close to the horizon that humans plan their movements under, when thinking about "go over there" plus the consequences. |
If that's how it works, that makes sense.
Anyway, I think it would be interesting to see how playfun would work in a football (soccer) game. I'm sure there are some for the NES, and it might be cool to see how it decides its strategy. Not sure though if it would be able to understand the end result of scoring a goal.
Keep up the awesome work ;) |
Hey. So. I got the binaries for learnfun and playfun but learnfun keeps spitting up that error, "check:!game.empty()" (or something like that.) Also, the config.txt would always show a !movie.empty() error until I removed the ".txt" extension from the file, changing the problem from the movie to the game.empty. I have config set up
game mario
movie mario-1.fm2
So I am at a loss for what I'm doing wrong. Any pointers? |
| Config looks right to me. You might just need a newline after the "movie" line; I don't remember if I uploaded that bugfix in the binaries. If you needed to remove ".txt" it might be because windows is hiding extensions from you (and it was actually config.txt.txt), so that might be creating other confusing scenarios. You can turn that behavior off in explorer or take a look using the command-line tools... |
| I got it after adding the newline, thanks! |
Uh, just kidding. Now that learnfun is sorted out, playfun gives me another error: "check failed: numworking > 0"
Any idea of how to fix? |
I believe that means you didn't run it with any helpers, so there are no processes to actually do the searching.
Sorry the error messages are so cryptic... |
*facepalm*
Thanks, it's working now! |
I feel like this is a dumb question, but where exactly is/should this config.txt file be?
|
Never mind that last one, I just removed the .txt from the one I made. Can't wait to see how it does with Galaga!
|
| Hehe... tool assisted speedrun bot... although I don't think this will set any speedruns (except for fastest playthrough by a bot) |
Don't know how often you check this, if at all, really.
I can't figure out how to get past the 'Check failed: !game.empty()' problem. Been trying a few things but have yet to figure out what I'm doing wrong. Any ideas? Thanks in advance. |
Anon from above here again. I finally managed to solve it. Basic mistake on my part.
Been running playfun for a little while now and I noticed it created a "backtrack replaced" and "replacement" .fm2 file. Mind explaining what that means? |
| When it does the backtracking step (explained in the paper), it saves the movie up to and including the part it's trying to replace, as well as what it replaces it with, in those files. Not usually very interesting. |
| Sorry for dumb question, but I'm getting the numworking error, what is a helper? |
| You need to run multiple copies of the program (one for each processor)-- these are helpers--and one main program to coordinate, which is the master. The README explains. |
How do I run multiple copies of the game with helpers? Sorry if you have already answered this.
|
(This isn't the same person as above)
I have my files, mario.nes, me-play-mario.fm2, and config.txt (hopefully, that's what you're supposed to do since I'm not really a computer guy), but it still gives me the "!game.empty()" error when I run learnfun. Config has a newline after the movie line, I tried running learnfun before AND after removing the ".txt" extension, nothing changed. |
| you need newline after game line too |
Tom, how i need to do this?
./playfun.exe --helper 8000
./playfun.exe --helper 8001
./playfun.exe --helper 8002
./playfun.exe --helper 8003
./playfun.exe --helper 8004
./playfun.exe --helper 8005
(wait until each helper says Ready.)
./playfun.exe --master 8000 8001 8002 8003 8004 8005 |
Tom, The Learnfun is not generating the .motifs!
please help. :( |
| Anonymous (50.125.74.153) , just wait) |
| Hmm, I don't understand why there's no motifs file? Does the game filename include weird characters? Is the input movie empty? |
| will it be better the more i record? and if i fail during recording will it think failing is a good thing? |
| If you still check this (and you probably don't), I am very new to this side of computers (setting up networks, compiling, etc.) and was wondering if you could explain the process from downloading learnfun/playfun (whether it's the source code or the 64 bit binaries, as I have 64 bit windows 7) to getting the software running playing games? I have the emulator and roms, can record an example for the software, but from there, the read me gets very complicated. Any help would be appreciated. Thanks Tom (or anyone else)! |
| UPDATE: I got everything to work up to the point where you have to run the "helpers" (What NUMW last asked about). How to I set up ANSI or whatever the software requires. I'm really looking forward to using the program, if I can ever figure it out! Thanks for any help! |
| UPDATE 2: I figured everything out. It wasn't as hard as I thought. I do have one question though... is there a way to continue running the emulation (let playfun pick up where I stopped it)? This would save much time and resources. Thanks again. |
| Glad you figured it out. :) There's currently no way to save and restore playfun's progress, though only because I didn't get around to it (it would be a pretty straightforward feature and I agree it would be useful). I'm sorry it isn't that user-friendly... |
The big things I found from this are:
1. The user (you, reading this now) needs to open up a blank Notepad and write four lines of code, of which lines 2 and 4 are left empty/blank...
-----code starts here------
game smb
movie smb.fm2
-----code ends here---------
...where "smb" is the name of the rom file (in this case, smb is Super Mario Brothers 1) and omitting the .nes filetype, and, a copy of the aforementioned recorded 1000-frame movie (in this case, smb-0.fm2 which I renamed smb.fm2 for my own purposes).
This file should be saved as "config.txt" in the same folder where learnfun and playfun reside.
2. The rom and the 1000-frame movie should also be copied (and named according to how you worded it in the config.txt file) into the same folder where learnfun and playfun reside.
3. Using the Command Line program in the Start menu, the user needs to navigate to where the learnfun / playfun / config.txt folder is. Once there, simply type in:
learnfun.exe
...and watch the fun happen.
But, see, this is where things go a bit wonky for me. Now, with the motifs and objectives set and ready to go, I type in:
playfun --helper 8001
...and it runs some text, until the final line is given:
[8000] <-[1;36;40m Ready.<-[m
...and that's where it hangs. It just hangs and does nothing.
Suggestions? |
The helper won't do anything until a master connects to it. You have to do (in another window):
playfun --master 8001
You probably want to run more than one helper (also in different windows). About two per core on your computer. You just give the ports of each one to playfun like
playfun --master 8001 8002 8003
Also, if you run ansicon, then it'll show colors and stuff instead of the [1;36;40m garbage. :) Good luck! |
| Since playfun helper is run on what I assume is a local host port would it be possible to setup a network of computers running helpers on a lan? My best computer is pretty low powered but I have 5 computers around the house. |
Yes, it should work. You'll either need to modify the source and recompile in order to get it to connect to other hosts, or set up a proxy on localhost using some other software (that's probably easier).
Note that it assumes there will be no network failures, so putting it on an actual network may be a bit risky. |
| Could i have permission to post videos of your software on YouTube? |
| Sure, go for it! :) Please post/send links of anything interesting it does, and I appreciate a link to tom7.org/mario. |
| Will do. My goal is to let Playfun beat a game on its own and also explore it playing with cheating. |
I understand that in the videos you post that in the upper right corner the 3 graphs that have the blue, red, and green represent neutral, bad, and good things happening. What is the difference between the 3? Are they 3 different time frames? For 1 sec, 3 sec, and 5 sec? Or something else entirely.
Thanks
SSS |
Hmm, I have an idea along those lines too. I wonder if they are related. :) I look forward to seeing what you come up with!
The three different graphs are just comparing the current frame to the previous frame, then 10 frames ago, then 100 frames ago. Not sure how useful it really is. You can see how these are computed in the source to scopefun; the SaveAV function is what to look at and I hope it's not that complicated. |
| hey OP, so I started learnfun and it created all of the .motifs. Then I start running playfun.exe with all of its --helper and --master but once I start up playfun.exe it gives "Check Failed numworking > 0" and crashes cmd. Help? |
| This means it can't connect to the helpers. Run the helpers first and then run the master with the ports that you used for the helpers. Try just running one helper and then one master, to make sure you have the right idea. |
I think this will help ppl
https://www.youtube.com/watch?v=QD6ZTTpCU9o&feature=youtu.be
Please feel free to use this vid tom ;) |
| Thanks, many people have been asking for something like this so I'll send them to your video :) |
| I apologize for the bother. I am new to this, and I reall don't know where the config.txt file is. I searched it, and I got a bunch of .cc files, as well as one .h file. I'm a noob at this, so help would be appreciates, thanks! |
^SCRATCH THAT!^
Fixed that, now I get a numworking > 0 error.
I know this means I am not running with "helpers". What are those, and how do I run them? |
^
https://www.youtube.com/watch?v=QD6ZTTpCU9o&feature=youtu.be |
| Thanks you, you creator of mechinal LULZ |
| You're welcome! (: |
---------------------
Game mario
movie three.fm2
-----------------------
They are in the correct playfun folder.
I dont know the problem.
gyazo.com/2cfed9ad6b8919be183bb472c71940e5
(i couldnt put http...)
(its not an inappropriate pic i swear) |
| Try lowercase "game". What error are you getting? |
| Can I run Playfun with virtual memory? I have only 3GB of RAM and a dual core CPU, and I'd love to give it a try, because I'll be able to run it for like 40 hours straight. |
| Sure, you can try, though it if it starts swapping it will get awfully slow. Probably you could run OK with just a single helper. It does require a 64 bit machine though. |
When I try to run Playfun, it says:
ReadProto: Failed to read length (got 0) err 0.
Error reading result from port [random port number] for work #1!
(gotten from master) It doesn't crash Playfun or anything (actually it runs fine) but it is slower. Does the fact that I am running it through WINE on Ubuntu create this issue? |
| It's possible that wine is emulating the windows api in a way that my program (or sdl_net) isn't capable of handling (0-length reads?). I'm not sure, though; I've never tried it under those conditions. On ubuntu you might have some luck compiling from source. Redirecting the output to /dev/null instead of a terminal might help with the performance problem if it's from printing that repeatedly. |
When I try to run Playfun, it says:
ReadProto: Failed to read length (got 0) err 0.
Error reading result from port [random port number] for work #1!
(gotten from master) It doesn't crash Playfun or anything (actually it runs fine) but it is slower. Does the fact that I am running it through WINE on Ubuntu create this issue? |
| Woah, wait... why did it repost my comment??? |
| I think I'm getting this thing. So how do I, (without compiling anything) teach it a text based goal? The readme speaks of a "tasbot" thing but there is no tasbot.exe in the folder |
| If you want to write custom objective functions, put them in "mario.objectives" (if the game in config is "mario"). Each line has the floating point weight followed by the decimal memory offsets (0-2047) in descending importance. |
| How do I set up scopefun? |
This is what I get from scope fun before it crashes
WinMain.
Loaded ddraw...
Get command line...
Parse command line...
Init SDL
SDL initialized OK.
With no command line argument, assuming movie name
smb-playfun-futures-progress.fm2
.. from config file.
scopefun.cc:49. Check failed: stb_rgba
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information. |
| I may have forgotten to include "controller.png" and "controllerdown.png" with the binary distribution; that failing on that line means it wasn't able to load some graphic.. You can download them from subversion (click the source link on the home page) and put them in the directory with scopefun.exe. Note that scopefun is the least user-friendly of the apps, since I was just using it on a one-off basis to make some of the movies! |
| thks it fixed it rendering now |
I have several questions…
Considering the long times you’ve seen playfun pausing, including your megaman pausing (you said in your video that it was like 9-10 hours of pausing). Is there some sort of limit to playfun’s learning?
I’ve shown it several castlevania plays, constantly showing it how to go from the start to stage 4. Several runs of playfun have resulted in it at least making it to stage 2 but my most recent few runs of playfun have not resulted in it going up the stairs to stage 2. I know it knows how to reach stage 2, its gone there on its own, I know that, so why isn’t it using it even though it knows it causes the stage value to increase?
How do you know or discern the locations of the values of the in game stuff (where you manually set some of its objectives)? Do you have to read it with a special program or just hex editing? As an example, what numbers would I have to use to tell it to prioritize points, health, lives, and levels in castlevania? I can’t see where those values are.
Is there a way to tell playfun to modify some of the game like a game genie while running? I’d think of this because if playfun does better on its last life, then just alter your lives such that you only have 1 so it has no choice but to do things correctly the first time. Or would one have to mod the rom itself to produce that result?
Is there a way to tell playfun to try to produce the best results within an objective time frame or to consider itself at least restrained or judged based upon objective time? Perhaps objective time as an actual “objective”? Like, say, 5 minutes? When I say “objective time” I mean such that it tries to do the best it can within 5 actual minutes, not game minutes (this would result in “pausing” being bad since it would only stop game time, not the actual time).
Can playfun tell the difference between a bullet to the head (like your contra video) and the actual end and or “beating” of a game?
You once manually changed the goals of playfun so that it was more conservative with its lives, with the algorithms in place will it eventually do that on its own if run long enough?
It appears to issue commands up to 60 times a second (one per frame), can it be changed where it only tries to make a move…say…every 3 frames instead of each individual frame?
Thought I'd consolidate them instead of posting them over like a 2 month period.
Playfun is awesome! ...just want to find a way to help it progress better and more efficiently. |
So my config.txt file looks like this;
1.game Double
2.movie Dragon2.fm2
3.
And I keep getting the game.empty() error.
Output from cmd is as follows;
""
WinMain.
Loaded ddraw...
Get command line...
Parse command line...
learnfun.cc:145 Check failed: !game.empty()
this application has requested to terminate in an unusual way, etc.
""
Then I get a notification, by windows, that learnfun.exe has stopped working and allows me to either debug the program, or close it. Any thoughts as to how i can fix this?
Running windows 8.1 |
| I'd expect the game line to be "double.nes" since the ROMs usually have an extension. But since it's saying that it didn't find a "game" line at all, maybe the file is actually called config.txt.txt instead of config.txt? |
Im trying to run play fun. i have got all the way to starting the master line. learnfun worked fine and so did the helpers but once i run the master lin windows comes up saying playfun.exe has stoped working and the comand line says:
D:\playfun>D:\playfun\playfun.exe --master 8000 8001
WinMain.
Loaded ddraw...
Get command line...
Parse command line...
Init SDL
SDL initialized OK.
Starting FCEUX 2.1.6-interim svn...
Loading Pinball.nes...
PRG ROM: 1 x 16KiB
CHR ROM: 1 x 8KiB
ROM CRC32: 0x035dc2e9
ROM MD5: 0xad809323fe92d0e083bf77ccecae462e
Mapper #: 0
Mapper name: NROM
Mirroring: Horizontal
Battery-backed: No
Trained: No
Power on
Loaded 280 objective functions
Resize cache 10000 1000
OK.
Skipped 592 frames until first keypress/ffwd.
[MASTER] Beginning Pinball.
Parallel step with 40 nexts, 40 futures.
[........................................]
SDLNet_TCP_Open(127.0.0.1:8000): Couldn't connect to remote host
This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
this is what the helpers say:
WinMain.
Loaded ddraw...
Get command line...
Parse command line...
Init SDL
SDL initialized OK.
Starting FCEUX 2.1.6-interim svn...
Loading Pinball.nes...
PRG ROM: 1 x 16KiB
CHR ROM: 1 x 8KiB
ROM CRC32: 0x035dc2e9
ROM MD5: 0xad809323fe92d0e083bf77ccecae462e
Mapper #: 0
Mapper name: NROM
Mirroring: Horizontal
Battery-backed: No
Trained: No
Power on
Loaded 280 objective functions
Resize cache 10000 1000
OK.
Skipped 592 frames until first keypress/ffwd.
i dont know what the problem is please could some on help me. |
| when ever i run playfun.exe the windows crash sign commes up |
| fixed it windows fire wall was blocking helpers |
| Will there ever be a way to use a GPU instead of a CPU in the future? |
Hey tom i had an error with playfun it reads
SDLNet_TCP_Open(127.0.0.1:8000): Couldn't connect to remote host |
Anon1: I've tried this, but so far without any real success. GPUs don't seem well suited to executing NES instructions, unfortunately. I have some more ideas in the queue that I hope to get back to, though.
Oaklr: Are you running helpers first? It's saying it can't connect to one on port 8000. Some other comments on this page may show you what you need to do, but I also tried to write it clearly in the readme... |
| is there still no save feature at all? i want to have playfun learn mega man, but i don't quite appreciate it eating up my cpu 24/7 and i can't just have it running in the background bc it makes most programs lag |
I haven't modified that version since release. I recommend setting the priority of the helpers to the lowest class; I didn't have any problem playing games or whatever foreground apps when I was running it for days on end to make those videos.
There's a new version in development now which is easier to use, but saving is still a difficult feature to support, so it's unlikely, sorry! |
Hey Tom
I am having a blast with your AI program.
currently running Gradius as it is my favorite childhood game.
I will post the results here when done.
thanks |
Hey Tom7! Been a long fan of this project since it came out in 2013ish. Hope you keep at it, I still enjoy playing with learnfun and enjoy its interesting quirks to this day even!
When compiling I couldn't get it to complete in hopes to get the network portions of the program up and running. I also would like make a small side-note, I noticed the two controller images were updated (smaller size... maybe alpha removed?) causing scopefun not to play nice with them... luckily the older ones still work fine. Anyways im not a programmer by any means but this did get me super interested into getting into it, wanted to say thanks for putting so much work and effort into something for everyone to enjoy!
Anyways fails to compile at utils.o missing |
Oh, I bet that using zopflipng or pngcrush on the controller graphics (I did this for my whole repository) made it so they can't be opened by stb_image. I think I got some reports about this one. Anyway, scopefun is not worth end-users using... just load up the movies in FCEUX.
Compiling is not going to be easy. I recommend just using the binaries unless you want to make modifications. (These do have network support, at least localhost.) Not sure what "utils" is -- util.o? netutil.o? |
Ahh I wanted to try using about 20+ computers with the project to boost timings at the cost of a heavy electric bill hahah for time. Scopefun still works fine, tricky to encode the end result but it works.
As for utils.o I am 98% sure it failed with "not found" I asked around and weirdly enough I cant get a valid answer no matter where I checked. Other than that as-is the program still works just fine with the binaries, using a super modern Intel 7800 gives about 10 inputs in a 6 to 40 second range! On a single computer of course.
Ill give compiling another good attempt in another hour or so. Cheers! |
Is it possible to recompile the binaries as to use the AVX instruction set (Intel Processors) (I am guessing that this could maybe improve performance, although I must be wrong...) [Or perhaps something less advanced? SSE4.1??? I don't know...
I spent about 6 hours trying to recompile it in many different ways but the mess of cross-compilation and mingw64+cygwin was just too much for me to get around. Not even Ubuntu and a VM did the trick...
(Errors with protobuf, problems with windows libraries and etc...)
Also, is it possible to use VS to compile it? (or anything that doesn't require too many fiddling with bizarre compilation methods?) |
Just wanted to say thank you for this work and I'm having a lot of fun experimenting with it. You mentioned a new version in the comments, is that something you expect to finish?
Thanks again! |
|
|
|