For me it is just trial and error. Other advice I have heard here and just from my own experience:
- set up a dummy palette with black, dark grey, light grey, and white. Make sure your sprite looks good in gray scale. This way it will look nice with just about any palette that is set up the same way (ie: has a range of dark/bright colors).
- you don't have a lot of space to communicate much information. what is the best use of that? For my 16x16 RPG character, arms are only a few pixels. Sometimes it is just a matter of changing a pixel and then seeing how it looks zoomed out or in animation.
- Sometimes things looks great zoomed in, but look like garbage zoomed out (i.e., you make a detailed tile that looks great zoomed in, but is just way too busy when zoomed out and repeated dozens of times on the screen). Likewise, sometimes things look bad zoomed way in, but look great zoomed out. zoomed out is ultimately what is going to appear on the screen.
- Not sure what you mean by "too big", but because of the limitations of the NES, you're sprites are probably not going to be any bigger than 32x32. stereotypical RPGs use 16x16. Mystic Origins/Searches uses 24X16 I believe. For my survival horror demo, the sprite was 32 tall and 16 or 24 wide.
- With animations, I try to keep everything as still as possible and only move what needs to be moved. I found early on that way too much stuff was moving for, say, a walking animation. Sometimes because of how I moved the pixels, the walking animation would appear to have a pixel flicker on and off and it was noisy and distracting.
- take inspiration from what you see others doing. save images to your computer and look at how they are set up.
Not sure if any of that is helpful
