Lets talk about saving [4.1+]

So, a long time back I asked about the possibility of saving with Nesmaker.

The general consensus was that it was too far off, though Kasumi linked me to a library to get started in the right direction.
http://nesmakers.com/viewtopic.php?t=867

In the time since then, I have had a lot more practice with ASM, and I am not so scared to experiment a bit. That being said,
I still feel this is out of my skill level. There are quite a few things I still do not understand.

I have been digging around in the scripts, and it looks like Joe started to implement a save system starting from the same library that Kasumi linked.
You can find the necessary variables defined but disabled in the Zero Page Ram.

Joe may bring us an implementation of this in a future update, but it is not likely to be a priority for a while. I thought we may be able to figure it out through discussion
and experimentation.

Here is what I know so far:
- We have a set of variables defined but disabled in the Zero Page ram that are used to point to a target bank, and specific addresses to write to one byte at a time.
- Based on comments around these variables, it appears that the empty bank 18 was intended to be used for saving, whether it was to store the saving code there or the
save itself I do not know.
- The library that Kasumi linked from the nesdev wiki has macros and subroutines for:
- Check if the cartridge supports saving
- Verifying a byte at a specific memory location (Checking to see if the byte we are writing has the same value as one already written, so we can skip writing it)
- Write a byte to a memory address.
- It seems to depend on a subroutine I do not recognize. 'FlashRamPage'. I am not sure if this is a subroutine already defined in Nesmaker somewhere else, or if it is
a missing piece that needs to be made.
- Flash saving seems to depend on having the code that handles it executed from RAM.

Here is what I am guessing:
- Bank 18 was left blank either for the purpose of storing the code that handles saving, or to have space for the save itself (or both?)

What I know about saving (serialization) from working with other game engines:
- With a lot of emulators, you can save your games 'state'. This saves a snapshot of ALL the games data, usually allowing you to start back from the moment you saved. An NES cartridge, and most games do not work that way.
- To save your game you must 'serialize' its important data.
- An exact snapshot of everything that exists in your game is unreasonable and unnecessary.
- Instead you want to save only what data is necessary to appear to save your progress ex:
-Screen player is on
-Screen position
-Player health
-Player Score (and any other stats)
-Screen Triggers
-Any other necessary user variables you have created that their value needs to be saved.

So, in Nesmaker if we can get the saving working, I think this is the way we would go about it.
Go through and write all the necessary bytes (screen triggers will take up a lot of space at 32) to the memory location designated for saving.
When we load, through a menu option or whatever, we read each of these values from their location in the save data and store them back in their original variables. Then warp to the screen.
 

Kasumi

New member
You include the flash.asm (in whatever bank, it doesn't matter). Possibly fix it to work with asm6. (I don't know what assembler it's written for offhand). You swap the bank it's in in if it's not in the fixed bank. You JSR to CopyFlashRoutine which copies the subroutines to RAM. (My understanding is you cannot execute them from ROM because ROM can't be read from while it's being flashed, so any flashing routine must be run from RAM.)

You store the bank, and address you're using for save data and returnbank to the RAM given. (returnbank will be swapped in when the routine is done, so it should be the bank the code is currently in). You put CallEraseSector which will erase the 4KB in the bank and address provided. (It's totally possible to erase actual code, or anything else in the ROM. So... don't. Provide an address/bank for a 4KB segment that is used for nothing else.)

You absolutely must erase the entire 4KB sector any time you want to save new values because the write byte function can only clear bits, not set them. (If $8000 in the save bank is $00 and you try to write $FF to it, it will stay $00. If it was $FF, you could make it $00 without erasing the sector, but then you'd have to verify every byte before you wrote...)

For every byte you want to save after you have erased the sector, store the bank and address within the save data you want to change in the RAM given. Load A with the byte you want to write, and Y with 0. Then put CallWriteByte which will write that byte to that address in that bank.

Reading is much simpler. Swap the save bank in and read the address.

And, a caveat, because you have to erase the entire 4KB segment where saves are to save again, you have to copy the values out of that "ROM" into RAM, then rewrite them. Yes, every time. If the user happens to power off the console in between the sector erase and the writing of the new values, the save data is lost. You also almost certainly have to change the header to allow saving (the battery bit must be set, bit 1 in byte 6 of the header).

Another caveat, there's a limited number of times writing is possible per cartridge, and while the limit is high, it's still worth making sure the data you want to save is actually different before you commit to erasing the sector. To do this, compare the values in your save bank to the values you're about to write. If one is not equal, saving is worth doing. If all are equal, saving will do nothing except eat away one of the times you're allowed to save.

I can maybe add this to my mapper 30 CHR RAM swap example I made for Mihoshi, but the above is probably 100% of the required steps. That said, I don't know if all emulators will support this or anything else. Mapper 30 being a new mapper is often a finicky compatibility thing.
 

Kasumi

New member
A double post so that it's not missed. I did the thing. Sort of. This is a Mapper 30 ROM that saves on FCEUX and not Mesen. Does it save on your carts when flashed? You tell me, I don't have one. Assuming it does work, the steps I outlined above were indeed correct. (Hopefully it's not required to be 512KB in order to flash, that'd be annoying.)

It looks like this:
Lph3Zjs.gif


Every time you power off or reset, the number should go up. If it doesn't work, tell me what it does do! (The number always stays 0? Always stays 1? Gray screen? Something else?)

Edit: Forum attachment didn't work, hang on, hang on...

Okay: https://www.dropbox.com/s/3451vmtho4qs8ov/Mapper30%20Save.zip?dl=0
 

Bucket Mouse

Active member
The Dropbox doesn't seem to be working either; I can't get the download to start.
EDIT: The download doesn't work in Firefox but it does in Chrome.

I can confirm the counter works on my computer (using FCEUX). What is it saving to though? What is the suffix of an NES save file?

Regarding the "limited number of saves" on a Flash cart...I read that number is somewhere past 200,000. The cart itself will wear out before the save does,
 

Kasumi

New member
200,000 is high, but not unfathomably, unreachably high. A mistake in programming could probably write more than 60 times a second, 3600 times a minute, 216000 times an hour. It's not likely to hit in normal use, but it's also not impossible over a large enough span of time. It's worth mitigating. For older games, the battery could be replaced. For these, when it's over it's over.

In FCEUX, it saves in the savs folder within the FCEUX folder with the same filename as the ROM except with extension ".sav" instead of ".nes" (by default, I think) like most other ROMs. So this would be "main.sav", assuming you did not change the filename. However, the format is different. Most NES games save to a designated portion of RAM, and the .sav files are just a direct copy of this RAM (8192 bytes). For this, FCEUX saves a byte for every byte in the ROM (since every byte in the ROM can be modified) as well as what seems to be a 32 byte header. It appears to only save changed bytes so that distributing a save is not basically distributing the ROM.

On the cartridge itself, it'd save to the cartridge itself. It essentially does the same thing the actual flasher hardware does when you're writing a new game. You could totally make a game that erases itself when the player gets a game over or something.
 
A game that erases itself when you get a game over? While I admit that is an interesting concept I feel that is far to cruel to the player :)

Anyways thank you for all of this information. I feel it is going to take me a little experimentation to understand, but I feel I have a much better idea of what to do now.
 

Kasumi

New member
Okay! I talked with sour, and apparently Mesen doesn't implement the self flashing behavior, so it's no surprise it doesn't work on it. But that honestly doesn't bode well for NES Maker save emulator compatibility if FCEUX is the only thing that currently supports it. <_<

But yo! Shoutouts to functionalform who tested this on hardware (a 512 KB version I kicked out since that works better with the flasher software I guess.). It works! But the real hero is whoever wrote that flash.asm library on the wiki. I think rainwarrior? Anyway lines 49-92 are the save stuff, the rest was for Mihoshi and CHR bank switching.

At the start of the game, copy the values from "ROM" to RAM. To save, copy the routines to RAM with jsr CopyFlashRoutine. Then erase the sector (CallEraseSector) (be sure to give the right bank and address.) Then CallWriteByte for every byte you want to write with the proper value you want to save. (You only need to erase the sector before each individual save, not each individual byte in that save.)
 

Mugi

Member
Sour is pretty good guy though, he'll propably implement it if you ask him to. (assuming you didn't already.)
 

dale_coop

Moderator
Staff member
Interesting, thank you Kasumi for all your help on that.
Now, we 'd need to find how to implant all that in NESMaker.
 

dale_coop

Moderator
Staff member
flash.asm is a script you can download following Kasumi's link.
I tried, I could make it work in NESMaker yet...
I will continue my tests next week (I hope someone can make it work and share a guide how to implant this... in NESMaker).
 

Kasumi

New member
In NES Maker is not really different. Include flash.asm in the fixed bank. After include "reset.asm" add:
Code:
ldy #$18 (or some other unused bank)
jsr bankswitchY;to swap in what will be the save bank.

;To load, lda $8000 for the first variable, lda $8001 for the second variable, etc. Up to 4KB. You'd then store their values someplace in RAM (you already know how to do that bit).

;For example:
lda $8000
sta curLives
lda $8001
sta curLevel

If you want to give the variables in "SRAM" (Flash ROM, really) names, open Basic/BankData/Bank18.asm, and do the following:
Code:
.org $8000;Technically already handled outside this file, but makes what's happening more clear
SaveSegmentStart:
curLivesFlash:;Name of the first variable
.db $03;Initial value of the first variable
curLevelFlash:;Name of the second variable
.db $00;Initial value of the second variable

;Add your own values here
.pad $8FFF
userPoweredOffFlash:;This MUST be the last value saved to. It is used to detect if the user turned the game off while we were saving
.db $00
And then the load in the fixed bank after reset would look like:
Code:
ldy #$18 (or some other unused bank)
jsr bankswitchY;to swap in what will be the save bank.

lda curLivesFlash
sta curLives
lda curLevelFlash
sta curLevel

lda userPoweredOffFlash
cmp #$FF
bne saveDataIsFine
;If here, the user turned off the power during the save
;So we need to give safe values to the variable we tried to save
;Example
lda #$03
sta curLives
lda #$00
sta curLevel

saveDataIsFine:
Your game would then use and update curLives and curLevel as it would any other RAM.

To save, make a subroutine somewhere in the fixed bank.
Code:
SaveSubroutine:
lda #$00;Disable the NMI
sta $2000;We really don't want it running and swapping banks
;or interrupting the very specific sequence of writes saving requires
;You may want to disable rendering too, I dunno

jsr CopyFlashRoutine;Copying the flash writing code to RAM (it can't run from ROM)

lda currentbank
and #%00011111
ora chrRamBank
sta ReturnBank

;We have to erase the full 4KB to be able to write new values
lda #$18;The Save Bank
sta TargetBank
lda #<SaveSegmentStart;Low byte of address where save bank starts
sta TargetAddress
lda #>SaveSegmentStart;High byte of address where save bank starts
sta TargetAddress_h
CallEraseSector

ldy #0;Start 0 bytes away from target address
lda curLives;Load the byte to write
CallWriteByte

ldy #1;Start 1 byte away from the target address
lda curLevel;Load the byte to write
CallWriteByte


;This allows for checking if the user turned off the game while saving happened.
;This MUST be the last byte saved or that check won't work
ldy #0;Start 0 bytes away from the target address
lda #<userPoweredOffFlash;Low byte of address where save bank starts
sta TargetAddress
lda #>userPoweredOffFlash;High byte of address where save bank starts
sta TargetAddress_h
lda #0;Something other than $FF
CallWriteByte

;Reenable rendering
lda #%10010000
ora showingNametable
sta $2000
rts

Final Step. In Flash.asm, change "FlashRamPage .EQU $0100" to "FlashRamPage .EQU SpriteRam". NES Maker is using the beginning of the stack for music already, and there's not enough contiguous free space in RAM anywhere else. ($BA contiguous free bytes in RAM are needed for the flash functions.) So, saving will corrupt the displayed sprites until the engine rewrites them. But you can also only jsr to SaveSubroutine in places where they aren't being displayed.

Secondary caveat, if you jsr to SaveRoutine when the NMI is already disabled, you should remove the stores to $2000 (otherwise calling this will reenable the NMI, which might break stuff that runs after it.)

Edit: More caveats: This uses $F8-$FD in zero page RAM. If NES Maker ever uses those, this breaks. But you can change the values in flash.asm itself, by changing the EQUs. TargetAddress/TargetAddress_h MUST be contiguous and MUST be in the zero page. SourceAddress and SourceAddress_h don't seem to be used, but same for them. (MUST be contiguous and MUST be in the zero page.)

Final Caveat: The user can reset or turn the game off at any time. Suppose they do it immediately after the flash ROM is erased. Then all the save variables are #$FF. If you wanna make a truly stable game, you have to check for that stuff. But to make that happening less likely, you also want to save only on user action, rather than all the time. I can't really give you advice on when is a good time to call SaveSubroutine and a good method for checking if the user turned the power off on the last save. It's just... complicated programming stuff. The "easiest" method is to add a variable for it. UserPoweredOffFlash, and make it the last variable saved. If it's $FF, the save data is invalid and needs to be reset.

Edit2: I guess... I'll just add that last part to the post.

Edit3: Also be aware FCEUX is the only emulator that supports this that I'm aware of. No, not even Mesen does right now.
 

dale_coop

Moderator
Staff member
Interesting, thanks Kasumi for those informations...
(Will try that maybe this weekend or next week)
 

dale_coop

Moderator
Staff member
Ok, I tried to implement that...
The first 2 or 3 steps was... quite ok. then I got lost.
Sorry guys, am still learning here. bank swtiching and specific memory adresses are still the Twilight Zone for me. I can spend a lot of time in there, without understanding anything, ...maybe some pieces.
Would need a step by step guide (line 196, just after that specific line... paste that exact code...) Then I will figure out and learn how all this work alltogether (my poor english skills might be a part of the problem, here).

Anyone got good results in NESMaker?
 

jorotroid

Member
So I sort of skimmed this thread because I'm not near ready for adding saving to my game yet, but I was a bit curious what was the latest on this. And one thing sort of overlapped with what I've been working on, so I thought I would share. In his last post, Kasumi suggested putting the FlashRamPage where the sprite RAM is located because there wasn't another location that had enough continuous free memory to hold the flash functions. He also said that doing this will corrupt the displayed sprites until they get rewritten again. I think on the Facebook group, Joe once mentioned that the page starting at $0700 was intended for this, but for the scrolling code he made, he had to use that ram to hold a second collision table. With some caveats, I think that location would still be best for putting the flash functions. First of all, if you are using the NoScroll core, chances are that page of ram is not getting used anyway, so you should be fine to use it. I haven't used the NoScroll much, but I feel it's a pretty safe bet that the NoScroll Core doesn't currently use that ram. As for the 4.1 basic core that does include scrolling, second collision table gets used on odd column screens (with the first column being 0, so even is first). So if you restrict saving to single even screens, or even were to make a special screen for saving where collision is not an issue, then the $0700 page should work fine without corrupting other data.

Not a plug for the scrolling core that I am slowly working on, but maybe also worth mentioning. I am reworking how collision is handled slightly so that I only need one page for collision instead of two, so that page will be freed up to do saving and maybe some other things that won't interfere with saving.
 

Kasumi

New member
I believe Flash should absolutely overlap with something else, because giving up $BA bytes of RAM just to have saving is not a good trade. If you freed $0700-$07FF for your scrolling core, great! That's 256 bytes you can use for not saving if you use the sprite RAM for saving. Saving only when sprites are not visible is a better trade. Or, you modify the NMI such that it doesn't sprite DMA ($4014) while the flash saving routines are in RAM and get the best of both worlds. Even if you don't do anything extra, all the sprite data is rewritten every frame, which is not true for all the collision data.

Off topic, be careful using only one page of RAM for collision. Here's why:
1GR6u82.gif

With exactly one page of RAM, (assuming 16x16 metatiles), there's not an offscreen buffer. So assuming the camera is right at the the screen boundary, objects could collide with the blocks on the opposite edge of the screen rather than what's actually offscreen. (Represented by green in the GIF. Light areas represent what's currently in RAM.). Suppose the white object bounces off walls. It will bounce back right in the gif above, because at the wrap point, there's a solid wall. But if the camera scrolled two pixels to the left, the ball would not bounce (because that scroll left would load that "empty" column back into RAM). (In the GIF the wrap point is really two screens wide, but the concept is the same)

Suppose you have exactly 16 columns in RAM. Now suppose the camera is at 0, 0. Now suppose the camera scrolls 8 pixels to the right. You have made a new column appear on screen without moving one of the 16 columns offscreen. (At both edges of the screen, you have 8 pixels of a column visible, and you also have 15 columns fully visible in the middle.) So if you have only 16 columns in RAM, except when the camera is at exact multiples of 16, you'll have 17 columns visible on screen.

Even if you upload a column at a time instead of a screen at a time, you'll still have this problem. You need some extra offscreen buffer, or you can destroy on screen objects that want to check wrapped data (which may look bad because they'd be mostly on screen), or you can just not care, but it's a concept that's worth being aware of. (Apologies for the info dump if you are already aware of these issues.)

Edit: Hmm, actually I guess there can be an offscreen buffer of one column, since columns are really only 240 pixels. I always forget this since mine are 256. :lol:
 

jorotroid

Member
Thanks for the additional insights, Kasumi

To the off topic topic, yeah, I thought up of two ways of dealing with reducing the RAM used for the collision data: a harder way and a lazy way. The hard way is pretty much what you described: having a column buffer to hold a little offscreen data, but because I'm not entirely sure where I'm going with other aspects of things, I decided to do the lazy way for now.

For the lazy way, I take advantage of the fact that NESmaker only allows for up to 16 tile collision types. This means for each byte of collision RAM, the high nibbles never gets used. So basically I just overlapped the two collisiontables. I saw two flavors of implementations I could go with to handle this: 1. each screen getting half the collision RAM page (each by representing two adjacent tiles), or 2. the low nibbles handle one screen while the high nibbles handle the other. I went with the latter for now because again it seemed like the quicker way to get things going. There is of course a drawback to choosing this direction. Accessing the low nibble is little faster than accessing the high nibble. I am not at a point to stress test this at the moment, but it seems good enough for now.
 

Kasumi

New member
It's six cycles slower to access. (and #%00001111 vs lsr a, lsr a, lsr a, lsr a). Even if you do it 100 times in a frame, It's only like 2% extra frame time. So don't worry about that.

Alternatively, it's two and eight cycles slower to access, which is still nothing. Indivisible uses 32 bytes of RAM for collision tables, with the consequence that destructible walls are a bit more of a pain to handle. (256 destructible things, at 1 bit per thing.) It looks up the tile from ROM rather than RAM, which isn't really slower.
 
Top Bottom