[Ohrrpgce] SVN: pkmnfrk/2436 so, I've decided against XML. Instead, I propose RELOAD, a new format.

[Mike Caron]

David Gowers wrote:
> Hi,
> This RELOAD is sounding a bit like Google Protocol-buffers. You might
> want to read up on it to see if you can use some of their research in
> this effort..
> Pertinent issues it deals with are compact
> forward-and-backward-compatible writing, streaming support, and
> security.
> Notably it seems to do a more efficient job of encoding 'common' tags,
> which might be adaptable here.
> 
> http://code.google.com/apis/protocolbuffers/
> "Protocol buffers are Google's language-neutral, platform-neutral,
> extensible mechanism for serializing structured data – think XML, but
> smaller, faster, and simpler."
> 
> http://code.google.com/p/protobuf/

I believe that the goal of protocol buffers are slightly different from 
that of RELOAD. I forgot to mention this when we were talking about 
converting RELOAD to and from XML, but I can see a point were we allow 
the game creator to embed an XML document with whatever document they 
want, and access it from plotscripting. The XML document would, then, be 
stored as a RELOAD document.

This would be impossible with the protocol buffers since it doesn't 
store tag names, it indexes fields by number.

> A few more misc comments follow. (+1 on the wiki)
> 
> 
> On Thu, Nov 6, 2008 at 6:59 PM, Mike Caron <caron.mike at gmail.com> wrote:
>> [2008-11-06 00:05:28] <tmc> OT: I want to make textbox text a binary stream
>> [2008-11-06 00:05:45] <MikeCaron> which would select the first textbox's
>> text
>> [2008-11-06 00:05:50] <MikeCaron> OT: ?
>> [2008-11-06 00:05:52] <tmc> we can use ascii codes < 32 for things like
>> changing font and changing font colour, and other codes
> 
> TMC: Why? You can have nice symbols showing those things, while
> actually inputting them and storing them as escape codes. If textbox
> text was binary, that would be optimizing for the minority case (how
> much special code characters vs how much normal characters are in a
> textbox?). There's a reason escape codes are so ubiquitous.

I'd like to point out that escape codes in any compiled language are 
also compiled. \n == 0x0a, \t == 0x09, etc.

>> [2008-11-06 02:10:09] <tmc> binary data
>> [2008-11-06 02:10:11] <MikeCaron> string
>> [2008-11-06 02:10:26] <MikeCaron> I almost added a blob type, before I
>> realized it was identical to a string
>> [2008-11-06 02:10:35] <tmc> strings: well... ok
>> [2008-11-06 02:11:07] <tmc> but arrays, I'm not sure about
>> [2008-11-06 02:11:12] <tmc> they may be more complicated than that
>> [2008-11-06 02:11:26] <MikeCaron> I'm fairly sure the existing format can
>> handle anything you throw at it
>> [2008-11-06 02:11:36] <MikeCaron> and, better than if you roll a sub-format
>> [2008-11-06 02:11:49] <tmc> add floating point though
>> [2008-11-06 02:11:54] <MikeCaron> or, at least, more flexibly
>> [2008-11-06 02:11:57] <MikeCaron> yes, that will be added
>> [2008-11-06 02:12:42] <tmc> what about shorts and ints (longs)
>> [2008-11-06 02:12:55] <tmc> you might not want to store everything in 4
>> bytes
>> [2008-11-06 02:13:05] <MikeCaron> well, I don't really see a need for a 2
>> byte int
>> [2008-11-06 02:15:47] <MikeCaron> I'll also add a 64-bit int, in case we
>> ever need it
> .. <variable-length int goes here>, yes?

A variable length int would be overkill. I think the current 
implementation is sufficient for 99.84% of all cases. If you really need 
larger than 64-bit, you're welcome to encode a BigInt as a byte stream.

>> [2008-11-06 02:23:30] <tmc> a null type?
>> [2008-11-06 02:23:39] <MikeCaron> eh...
>> [2008-11-06 02:23:42] <tmc> like dataless tags in HTML
>> [2008-11-06 02:23:45] <MikeCaron> I don't really see the need
>> [2008-11-06 02:23:47] <tmc> like <p>
>> [2008-11-06 02:24:16] <tmc> though such things would occur, won't they
>> [2008-11-06 02:24:17] <MikeCaron> I dunno
>> [2008-11-06 02:24:21] <MikeCaron> they might?
>> [2008-11-06 02:24:25] <tmc> I guess it would be a bitset
>> [2008-11-06 02:24:27] <MikeCaron> only if we encode text in RELOAD
>> [2008-11-06 02:24:36] <MikeCaron> see
>> [2008-11-06 02:24:44] <MikeCaron> that implies that the element itself is a
>> data item
>> [2008-11-06 02:24:53] <MikeCaron> which, I suppose is ok
>> [2008-11-06 02:24:59] <tmc> yes, that's the point
>> [2008-11-06 02:25:01] <MikeCaron> yeah, I think I will do taht
>> [2008-11-06 02:25:02] <MikeCaron> *that
>> [2008-11-06 02:25:10] <tmc> so how would you store bits
>> [2008-11-06 02:25:18] <tmc> only store on bits?
>> [2008-11-06 02:25:22] <MikeCaron> and, add it at the front of the list for
>> maximum renumbering
>> [2008-11-06 02:25:26] <MikeCaron> yeah
>> [2008-11-06 02:25:34] <tmc> each with a dummy byte as data?
>> [2008-11-06 02:25:37] <MikeCaron> no
>> [2008-11-06 02:25:40] <MikeCaron> with no byte
>> [2008-11-06 02:25:44] <MikeCaron> if it's null, it's nothing
> Cool, I like this encoding of bitsets. Far more inspectable, and
> reasonably efficient for moderate N of set bits.

Inspectable? Hardly anything of this format is inspectable.

>> [2008-11-06 03:00:07] <tmc> slurp?
>> [2008-11-06 03:00:23] <MikeCaron> slurp == read in
> 
> Perl references, hehe.

This is not an admission to ever having written any PERL.

>> [2008-11-06 03:00:27] <tmc> into what format?
>> [2008-11-06 03:00:40] <tmc> guess it doesn't matter
>> [2008-11-06 03:00:43] <MikeCaron> an internal format
>> [2008-11-06 03:00:47] <MikeCaron> that's all you need to know
>> [2008-11-06 03:08:55] <tmc> have you given any thought to lump
>> descriptions/editedit
>> [2008-11-06 03:09:00] <MikeCaron> not at all
>> [2008-11-06 03:12:10] <tmc> I guess that plan would be modified with the
>> RELOAD types
>> [2008-11-06 03:12:33] <tmc> this bit:
>> [2008-11-06 03:12:34] <tmc> Data field storage type
>> [2008-11-06 03:12:34] <tmc>     * Data field offset in record
>> [2008-11-06 03:12:34] <tmc>     * Data format (integer, tag number, bitsets,
>> strings, attack id, item id, hero id, enemy id, etc. Basically everything in
>> updateflexmenu() and more)
>> [2008-11-06 03:12:34] <tmc>     * Data field size (strings only)
>> [2008-11-06 03:12:35] <tmc>     * Data field BASIC type (INTEGER, STRING)
>> [2008-11-06 03:13:00] <MikeCaron> at this point, I don't want to think about
>> that stuff
>> [2008-11-06 03:13:05] <tmc> ok
>> [2008-11-06 03:13:12] <MikeCaron> it's too meta, even for me
>> [2008-11-06 03:13:17] <tmc> like I said, I wanted to start on this stuff
>> [2008-11-06 03:13:31] <tmc> but I guess I should finish something before I
>> start on anything else
>> [2008-11-06 03:15:13] <tmc> oh
>> [2008-11-06 03:15:18] <tmc> but did you see bug 92
>> [2008-11-06 03:15:40] <tmc> the idea to skip unlumping
>> [2008-11-06 03:15:56] <tmc> seems to me that nothing needs to be done to
>> start on this
>> [2008-11-06 03:16:14] <tmc> meh
>> [2008-11-06 03:16:21] <tmc> I'm going to leave now
>> [2008-11-06 03:16:26] <MikeCaron> ok
>> [2008-11-06 03:16:28] <tmc> exam, and all that
>> [2008-11-06 03:16:32] <MikeCaron> alright
>> [2008-11-06 03:16:33] <MikeCaron> good luck
>> [2008-11-06 03:16:48] <tmc> I think this was very productive
>> [2008-11-06 03:16:55] <MikeCaron> quite
>> [2008-11-06 03:17:07] |<-- tmc has left irc.esper.net (Quit: Client Exited)
>>
> 
> 
> David
>