[Ohrrpgce] better palette options

[David Gowers (kampu)]

On Mon, Oct 11, 2010 at 3:36 AM, Mike Caron <caron.mike at gmail.com> wrote:
> What about RELOAD (or, I guess, XML's) data model don't you like?
That nodes with children can also have values.
That's a pretty extraordinary behaviour for a tree data model, IMO.

Unless you expect people to XSLT their XML source data,
I think a model like YAML or JSON is much simpler while still
supporting the basic data model.

(you can currently convert YAML or JSON to RELOAD with a bit of work.
I'm just arguing that the "nodes with children can also have values"
is a specific, and probably unneeded, accommodation for XML's strange
behaviour.)

 I'm not
> trying to defend it, I honestly want to know what you don't like. Perhaps
> something could be improved?
>
>>> But it is now.
>>
>> I wasn't actually requesting it. RELOAD as container for animation
>> data, as in your proposal, seems like a fairly good idea. RELOAD as
>> container for pixel data may be overly heavy.
>
> I agree that pixel data should not be stored in a RELOAD container.
>
> Rather, all the animation data should be stored in a single RELOAD document,
> and the images separate from that.
>
>> RELOAD annoys me a
>> little too, due to much increased complexity of read/writing.
>
> But, surely, you agree that the benefits of having a strongly typed and
> named data format outweigh the complexity, right?
>
> Honestly, I think the lump resizing scheme for older lumps is a lot more
> complex than RELOAD...

>
>
> I think that's a really bad idea. The point of bounding boxes is that they
> are independent of the graphic on screen. Case in point:
>
> What if, on a frame with a small bounding box, a character moves up against
> a wall. Then, on the next frame, he has a large bounding box, which now
> intersects with the wall. Which of the following happens?
>
> 1. The character can now move into the wall?
> 2. The character is ejected (a la super mario bros)?
> 3. The character can only move out from the wall?
> 4. The character can't move at all?
> 5. The character is disintegrated since two particles cannot share the same
> quantum state?
>
> If the bounding boxes are fixed (or, at least, independent of sprites),
>

..?? Hello sentence which is not there.

>>>>
>>>> I don't understand the difference between the 'animset' command, and
>>>> chaining to another animation using 'end'.
>>>
>>> animset != anim.
>>> it doesn't set the anim, it sets the animset.
>>> Anyway I removed that by allowing varlength tuples.
>>
>> What varlength tuples?
>>
>>>>
>>>> You're also missing animation names. I think naming animations is
>>>> better than numbering them.
>>>> Then you can add a attackee flinch
>>>> animation to an enemy, say, just by adding an animation and naming it
>>>> "hurt".
>>>
>>> +1. I was thinking about this problem and couldn't figure out how to
>>> do it nicely, but this is excellent.
>>
>> But there's a complication, referring to animations by name may not
>> always be appropriate. Quoting from the 'text tags' thread, where I
>> proposed embedding tags in textboxes which trigger portrait
>> animations:
>>
>>>> Maybe animations should get both a name and a number. The name can be
>>>> used everywhere that it makes sense to use a name, and the number can be
>>>> used in whatever small number of places a name isn't practical (like
>>>> here)
>>>
>>> OK, sounds practical. But I wonder whether the ID number should be an
>>> index into the big global table of animation names (shared by all
>>> spritesets of any sort), or refer to the n-th animation currently
>>> defined for this spriteset. I'd prefer the first, since it might make
>>> sense to sort animations, say alphabetically.
>>
>>
>>
>>>>
>>>> The check/setctr codes seem to be very inefficient for creating loops:
>>>>
>>>> setctr ctr0 = x
>>>> label 1
>>>> ...loop contents
>>>> setctr ctr0 -= 1
>>>> checkctr ctr0 == 0
>>>> goto label 2
>>>> goto label 1
>>>> label 2
>>>>
>>>> Those last 5 instructions could be replaced with a loop(ctr,label)
>>>> code: decrement ctr and goto label if ctr>  0. A relative frame change
>>>> code would also be very useful for loops.
>>>
>>> oh right, ctr and label can fit into 16 bits easily(3+8)
>>> Good idea. (note that I'm leaving in the original ops -- they have a
>>> specific use that cannot be readily attained by a loop command :
>>> interleaving instructions, like 'wait 1' that only execute eg on>=
>>> second loop through a particular area
>>> of code.
>>
>> Your definition of loop isn't correct.
>>
>> The actual encoding of the bytestring, bits per opcode and argument,
>> isn't specified. Making them all 16 bit signed is easy, but we if want
>> to cram multiple arguments into the same short, maybe it would just be
>> simpler to same that opcodes are 8 bit, and arguments are either 8 or
>> 16 bit signed (or a string (wouldn't be necessary if we numbered
>> animations internally?)).
>>
>>>>
>>>> I suggest a 'reset' command. Changing the sprite state after the end
>>>> may be desired (ie. a transition animation), and sometimes not.
>>>> 'reset' would return to the original gfxset, frame, palette, and
>>>> offset.
>>
>> Wondering why you noted reset as only resetting gfxset  - you don't
>> think the rest is useful?
>>
>>>>> Note that I intentionally made it as unrestricted as possible.
>>>>> Imposing additional constraints could allow the removal of certain
>>>>> tuple types and runtime data fields. Whether additional constraints
>>>>> are appropriate is a matter for discussion.
>>>>>
>>>>> Comments on PNG:
>>>>> * I saw your suggestion, Mike, and I thought : though it isn't the
>>>>> only option, it has a lot going for it.
>>>>>
>>>>> * Note we may want to rewrite imported images even if they are in PNG
>>>>> already: Some programs (notably Photoshop) are pretty bad at
>>>>> compressing PNG relative to libpng.
>>>>>
>>>>> * I opted to include a simple source-tracking tEXT chunk
>>>>>  to aid import<->  export cycles
>>>>>
>>>>> * using PNG (and implicitly assuming 256-color sprite data) has the
>>>>> side effect that we can play nicer with eg Grafx2, which always writes
>>>>> a '256-color' image even though only the first 16 colors are used.
>>>>
>>>
>>>> I'm uneasy about using PNG:
>>>> * the biggest problem: paletted PNG files require a palette, which is
>>>> useless to us. We could instead encode our images as 4-bit/8-bit
>>>> greyscale pngs, but then the PNG is not even human-viewable; isn't
>>>> that half of the point of using PNGs? Would we put real palette
>>>> information (default palette) in a private chunk?
>>>
>>> I wasn't sure what to think about this. Because default palettes of
>>> the 256-color variety are NOT addressed with the current scheme.
>>>
>>> Although it looks like we can use negative values in defpal%d.bin to
>>> implement that.
>>
>> I'm wondering how much trouble it would be to unify 256 and 16 colour
>> palettes. Either allow palettes to have a variable number of colours,
>> or upgrade all palettes to 256 colours so that there is no
>> distinction. It would make a lot of things easier.
>>
>>> Anyway I changed to a reload-wrapped blob of pixels.
>>>>
>>>> * it would be a horrible idea to import whole PNG files. There could
>>>> be all kinds of other things in them that could change meaning later
>>>> (eg. requesting pixel data from libpng in a different way)
>>>>
>>>> * requires interfacing with an external library, all this is going to
>>>> be potentially much more work than using our own barebones file
>>>> format. However we might want to use libpng for importing/exporting
>>>> png files anyway.
>>>
>>> +1 on that.
>>>>
>>>> * It would probably actually increase file size when used for
>>>> walkabout sprites and smaller: chunk headers add a minimum of 56 bytes
>>>> (assuming no palette chunk) without even encoding the image size. And
>>>> individually compressing each 20x20 frame may yield lower compression
>>>> than just using zlib on the entire RPG file/sublumped spriteset file.
>>>
>>> Yeah, I've tested this. It does inflate small image files. we start
>>> winning at about 24x24 (288 raw bytes vs 158 compressed png).
>>> as for (individual) zlib:
>>>
>>> $>  gzip -9 *.bmp; gzip -l *.bmp.gz
>>>
>>>         compressed        uncompressed  ratio uncompressed_name
>>>                329                1518  81.4% jbokei attack 2 frame 2.bmp
>>>                174                 246  49.2% jbokei boxborder 0 frame
>>> 0.bmp
>>>                690                3310  80.5% jbokei largeenemy 0-.bmp
>>>                604                7478  92.5% jbokei largeenemy 0.bmp
>>>                595                3318  83.4% jbokei largeenemy 0+.bmp
>>>               1007                3318  70.9% jbokei largeenemy 1.bmp
>>>               1016                7478  87.0% jbokei largeenemy1.bmp
>>>               1069                3318  69.0% jbokei largeenemy 2.bmp
>>>                482                1518  71.1% jbokei mediumenemy 0.bmp
>>>               1372               65078  97.9% jbokei screen0.bmp
>>>                300                 798  67.7% jbokei smallenemy 0.bmp
>>>               3276               65078  95.0% jbokei tileset0.bmp
>>>                189                 358  60.9% jbokei walkabout 0 frame
>>> 0.bmp
>>>                156                 406  72.9% jbokei weapon 0 frame 0.bmp
>>>                543              192054  99.7% nacmene.bmp
>>>                583                3318  83.2% p0.bmp
>>>              12385              358592  96.6% (totals)
>>>
>>> compressing as a block yields
>>> 11593
>>>
>>> actual compression should be better, though, since we would have less
>>> header
>>
>> I also compared compression of pngs (produced with Gimp) with bmps run
>> through gzip -9 on backdrops from Vikings and found that the pngs were
>> consistently smaller, but usually by only 1%. The gain from
>> compressing a tar of 4 bmps was only about 4% after accounting for
>> size increase due to tar headers.
>>
>>>>
>>>>> On another topic:
>>>>> * my proposition does not include backgrounds or tilesets.
>>>>>  I believe backgrounds can fit into the scheme in a natural way.
>>>>>  I'm not sure what to do with tilesets as I understood that some
>>>>> other upgrade relating to them is incipient, but not sure exactly
>>>>> what.
>>>>
>>>> Tilesets should allow a variable number of tiles. Tile animations
>>>> should be more flexible. We might as well not limit ourselves to 20x20
>>>> tiles. Beyond that, no plans. We could store them as an array of
>>>> individual tiles, but import/export/display in the current format. I'm
>>>> not sure if there's any benefit.
>>>
>>> Okay, there was another change I needed to make anyway
>>> which should address variable length and size other than 20x20.
>>> Tile animation (might, possibly) be able to be a subset of the
>>> proposed sprite anim format, including these ops: (relframe wait label
>>> goto loop setctr checkctr reset end editpal cyclepal)
>>>
>>> This would be a superset of the current TAP.
>>
>> Oh, I completely forgot; I was wondering whether that would be possible.
>>
>> One possibility would be to have an animation state for each tile in
>> each tile animation range (there are 256 - 160 = 96) for each tileset
>> used on the map. Upgrade the existing 2 tile animation patterns to 2
>> animations plus 96 animations which set the starting tile and then
>> chain to one of the two translated patterns. I really don't like the
>> sound of this.
>>
>> Much more reasonable would be to let people associate with each
>> animation for a tileset the range of the starting tile (frame) - a
>> small tileset-only extension to the file format. So TAP animations
>> would be translated to two animations for the first tile in each
>> range, with the ranges set to 48. People could define animations with
>> ranges of 1 to create a single tile animation.
>>
>> Note that the frame number should wrap around for TAP animations to work.
>>
>> Storing each tile as a separate frame is a slight nuisance. I'm just
>> going to load them all into a single frame as an optimisation.
>>
>>>>
>>>>> swap in/out sets of 16 to recolor things easily.
>>>>
>>>> I really like the idea of treating a 256-colour palette as 16
>>>> 16-colour palettes. But swapping 16-colours individually doesn't sound
>>>> simple to support.
>>>
>>> Do you mean from a UI perspective?
>>
>> Partially; I was thinking of associating with each sprite an array of
>> palette indices, rather than giving each sprite its own copy of its
>> palette. Using copies of the palette will . It's my nature to be
>> uneasy about the extravagance of a 256 byte array per sprite, but we
>> can use copy-on-write, and anyway 256 bytes
>>
>>> http://bpaste.net/show/10014/
>>> ^ current draft of proposition. addresses many of the issues brought up.
>
> I have my own thoughts on the structure of all this, but I will put it in a
> separate email, since this one is getting hella long.
>