[Ohrrpgce] State of nightly builds

[Ralph Versteegen]

On Sun, 17 Jan 2021 at 02:36, Quasar Nebula <towerofnix at gmail.com> wrote:

> just lurking! :)
>
> may i ask what the function of sourceslices.bi is? i skimmed its contents
> as well as sourceslices/update.sh, and the best i can guess is that its for
> converting default (or perhaps internal) data files of some sort! im
> curious how far off that guess is c:
>
> ~jade
>

sourceslices/*.slices are slice collections that are used by Game (for the
builtin slice-based menus like Items) and they are converted to
sourceslices.bi by sourceslices/update.sh which calls slice2bas, which gets
compiled into Game. This is a nuisance because it takes a long time to
compile. What we should do instead is embed the .slice files into the
executables. I already started on that. (Slice collections used by Custom
are instead stored in data/ and distributed as separate files)


> On Sat, Jan 16, 2021 at 12:24 AM James Paige <Bob at hamsterrepublic.com>
> wrote:
>
>> Yes, they are still built in sequence. They are on the same machine as
>> they have been for about 3 years. They max out the cpu in sequence, I don't
>> know what would happen in parallel.
>>
>
The fact that they now take over 3 hours to build but always took about an
hour in the past is pretty suspicious then. I don't think anything has
changed in the build system that could have caused it, and the compilers
haven't changed. Maybe the CPU is overheating and throttling down?

>
>> I have not switched the Linux builds over to Docker yet. I'll do that
>> when I find time to confirm that they are complete and correct.
>>
>> I actually wonder if we are deleting state that doesn't need to be
>> deleted. It does kinda seem like it is doing a full rebuild every time. I
>> have to check the scripts.
>>
>
The Windows and Mac nightlies do a full rebuild because they do multiple
builds with different args passed to fbc (debug vs non-debug builds on
Windows, 32 vs 64-bit on Mac). Because 32 and 64-bit Linux nightlies are
separate VMs they only rebuild (at minimum) the modules that include
ver.txt. SConscript actually supports keeping .o files for different builds
in different subdirectories, so that it can switch between them without
having to delete all built objects. I should try setting that up. I
frequently compare linux and windows or 32 and 64 bit builds so I find the
full rebuild annoying.


>
>> I do know the Android builds do delete all state and rebuild from scratch.
>>
>
This is trickier to avoid. I don't understand what's going on. On the
nightly build machine it looks like it does a full rebuild of libfb, SDL,
SDL_mixer, vorbis, ogg, mikmod, timidity, and more, and even libraries we
don't use like libpng and libjpeg, while on my computer sometime it does a
full build of (at least some of) these and sometimes only a partial rebuild
of SDL and a full rebuild of SDL_mixer. Odd.


>> Also, Android nightly builds are in a VM too, currently.
>>
>> Thumbs up for getting rid of sourceslices.bi
>>
>> A debian arm package sounds like a great idea. I do like the idea of
>> supporting easy installation on Rasbian
>>
>> On Fri., Jan. 15, 2021, 10:53 p.m. Ralph Versteegen, <teeemcee at gmail.com>
>> wrote:
>>
>>> Looking at http://hamsterrepublic.com/ohrrpgce/nightly/ I see that
>>> nightly builds are timestamped over the course of more than three hours.
>>> Are they still built in sequence? On a much slower machine than previously?
>>> Are the docker images now used for the linux (or any other) builds, or do
>>> they still all use VMs except Android? Are the nightlies recompiled from
>>> scratch every night or is the state from the previous compile there so
>>> scons can avoid recompiling?
>>>
>>> Incidentally I've been meaning to  reduce the number of modules that
>>> need to be recompiled when the backends or date/svn revision change. I've
>>> also been meaning to get rid of sourceslices.bi. That'll reduce compile
>>> times substantially.
>>>
>>> BTW, are there any plans for a Debian ARM package? I assume we could
>>> create a .deb that could be installed both on an RPi and a Debian ARM
>>> machine.
>>> _______________________________________________
>>> Ohrrpgce mailing list
>>> ohrrpgce at lists.motherhamster.org
>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>
>> _______________________________________________
>> Ohrrpgce mailing list
>> ohrrpgce at lists.motherhamster.org
>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>
> _______________________________________________
> Ohrrpgce mailing list
> ohrrpgce at lists.motherhamster.org
> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20210117/4e7139a0/attachment.html>