[Ohrrpgce] State of nightly builds

Sat Jan 16 19:30:20 PST 2021

On Sat., Jan. 16, 2021, 7:04 p.m. Ralph Versteegen, <teeemcee at gmail.com>
wrote:

>
> On Sun, 17 Jan 2021 at 02:36, Quasar Nebula <towerofnix at gmail.com> wrote:
>
>> just lurking! :)
>>
>> may i ask what the function of sourceslices.bi is? i skimmed its
>> contents as well as sourceslices/update.sh, and the best i can guess is
>> that its for converting default (or perhaps internal) data files of some
>> sort! im curious how far off that guess is c:
>>
>> ~jade
>>
>
> sourceslices/*.slices are slice collections that are used by Game (for the
> builtin slice-based menus like Items) and they are converted to
> sourceslices.bi by sourceslices/update.sh which calls slice2bas, which
> gets compiled into Game. This is a nuisance because it takes a long time to
> compile. What we should do instead is embed the .slice files into the
> executables. I already started on that. (Slice collections used by Custom
> are instead stored in data/ and distributed as separate files)
>
>
>> On Sat, Jan 16, 2021 at 12:24 AM James Paige <Bob at hamsterrepublic.com>
>> wrote:
>>
>>> Yes, they are still built in sequence. They are on the same machine as
>>> they have been for about 3 years. They max out the cpu in sequence, I don't
>>> know what would happen in parallel.
>>>
>>
> The fact that they now take over 3 hours to build but always took about an
> hour in the past is pretty suspicious then. I don't think anything has
> changed in the build system that could have caused it, and the compilers
> haven't changed. Maybe the CPU is overheating and throttling down?
>

Hmm... It just so happens that I have Prometheus Node Exporter set up on
the nightly build box, so I can see, yes, something is definitely wrong.

(Screenshot attached)

Something started using up CPU at the beginning of the week when I moved
the nightly build box to another room. I'll check it out...

Yes, something is wrong with the local Minecraft servers that I run on the
same box for my kids. They are supposed to go idle when not being used, but
they are all spinning at 25% cpu  each....

Okay, I better kill and restart those :D

Let's see if tonight's build goes smoother

>>> I have not switched the Linux builds over to Docker yet. I'll do that
>>> when I find time to confirm that they are complete and correct.
>>>
>>> I actually wonder if we are deleting state that doesn't need to be
>>> deleted. It does kinda seem like it is doing a full rebuild every time. I
>>> have to check the scripts.
>>>
>>
> The Windows and Mac nightlies do a full rebuild because they do multiple
> builds with different args passed to fbc (debug vs non-debug builds on
> Windows, 32 vs 64-bit on Mac). Because 32 and 64-bit Linux nightlies are
> separate VMs they only rebuild (at minimum) the modules that include
> ver.txt. SConscript actually supports keeping .o files for different builds
> in different subdirectories, so that it can switch between them without
> having to delete all built objects. I should try setting that up. I
> frequently compare linux and windows or 32 and 64 bit builds so I find the
> full rebuild annoying.
>

That option of separate build subdirectories sounds super useful! :)

>
>>
>>> I do know the Android builds do delete all state and rebuild from
>>> scratch.
>>>
>>
> This is trickier to avoid. I don't understand what's going on. On the
> nightly build machine it looks like it does a full rebuild of libfb, SDL,
> SDL_mixer, vorbis, ogg, mikmod, timidity, and more, and even libraries we
> don't use like libpng and libjpeg, while on my computer sometime it does a
> full build of (at least some of) these and sometimes only a partial rebuild
> of SDL and a full rebuild of SDL_mixer. Odd.
>

The Android builds does do both a 32 bit and 64 bit build on the same vm,
and I am pretty sure it cleans everything before each

>
>>> Also, Android nightly builds are in a VM too, currently.
>>>
>>> Thumbs up for getting rid of sourceslices.bi
>>>
>>> A debian arm package sounds like a great idea. I do like the idea of
>>> supporting easy installation on Rasbian
>>>
>>> On Fri., Jan. 15, 2021, 10:53 p.m. Ralph Versteegen, <teeemcee at gmail.com>
>>> wrote:
>>>
>>>> Looking at http://hamsterrepublic.com/ohrrpgce/nightly/ I see that
>>>> nightly builds are timestamped over the course of more than three hours.
>>>> Are they still built in sequence? On a much slower machine than previously?
>>>> Are the docker images now used for the linux (or any other) builds, or do
>>>> they still all use VMs except Android? Are the nightlies recompiled from
>>>> scratch every night or is the state from the previous compile there so
>>>> scons can avoid recompiling?
>>>>
>>>> Incidentally I've been meaning to  reduce the number of modules that
>>>> need to be recompiled when the backends or date/svn revision change. I've
>>>> also been meaning to get rid of sourceslices.bi. That'll reduce
>>>> compile times substantially.
>>>>
>>>> BTW, are there any plans for a Debian ARM package? I assume we could
>>>> create a .deb that could be installed both on an RPi and a Debian ARM
>>>> machine.
>>>> _______________________________________________
>>>> Ohrrpgce mailing list
>>>> ohrrpgce at lists.motherhamster.org
>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>
>>> _______________________________________________
>>> Ohrrpgce mailing list
>>> ohrrpgce at lists.motherhamster.org
>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>
>> _______________________________________________
>> Ohrrpgce mailing list
>> ohrrpgce at lists.motherhamster.org
>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>
> _______________________________________________
> Ohrrpgce mailing list
> ohrrpgce at lists.motherhamster.org
> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20210116/12e3454c/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot_20210116-220409.png
Type: image/png
Size: 133997 bytes
Desc: not available
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20210116/12e3454c/attachment-0001.png>