[Ohrrpgce] CrashRpt report summaries

Mon Aug 19 08:03:07 PDT 2019

I think it would be redundant to open one issue per report when most would
just immediately be marked as duplicate, since I can see at a glance that
they're a known bug. Nicer not to have hundreds of uninteresting closed
issues. But I guess that's just being picky, plenty of projects do manage
with spam-filled bug trackers.

More importantly, I think a table (with sortable columns) is just the best
way to view large numbers of report summaries. For example I could quickly
see which bugs have identical or similar backtraces, whether all reports of
a bug are coming from the same person or the same game, which were reported
in a short period of time (we have some Game+Custom dual crashes), if
someone reported a bunch of bugs but only provided their email/name once,
what are the oldest and newest build for which a bug has been reported. And
that's something on which github's issue tracker falls flat compared to
bugzilla: it doesn't seem that you can sort by anything like that, and you
can only attach labels, not non-categorical data, so you can't tabulate.

It is odd to use a wiki in this way though. Oh, I just remembered GH has a
wiki too. But it seems it doesn't have tables with sortable columns.
Another option would be something like a php/python/etc script that serves
a simple html page with a table generated from a text file, wiki page, or
the github api.

Things I imagine might be hand assigned to each crash report in the table
(rather than in the issue tracker):
-bug number
-whether the bug is fixed
-whether the minidump has been inspected with a debugger (aside from seeing
local variables and threads, often you get a better backtrace)
-whether the debug logs have been sighted for interesting messages
More detail belongs in an issue tracke.

But the privacy issue is a question regardless of whether we put them on a
wiki or elsewhere that's public.

Oh, also CrashRpt has a sister project, CrashFix, which is web server
software for receiving and managing the crash reports, with a built-in
issue tracker, permissions/user management and its own utility for
generating backtraces (which is quite a major undertaking). But like
CrashRpt the original project isn't maintained, now there are a lot of
forks. Initially I thought it was too heavy for what we needed (it requires
a daemon running to process reports) and I didn't manage to get working,
since I don't know what I'm doing when it comes to servers. I didn't
anticipate spending quite this much time on my own solution. However, it
hasn't been a reinvention of the wheel, there's a lot of OHRRPGCE-specific
info in the report summaries from process_crashreports.py.

On Mon, 19 Aug 2019 at 21:50, James Paige <Bob at hamsterrepublic.com> wrote:

> This crash reporting feature is great. I just read through a bunch of them.
>
> I am not sure what is best re putting the reports in the wiki. Is the idea
> that we want to be able to mark them resolved? Would it be better (if
> noisier) to put them in the issue tracker?
>
>
>
> On Sun, Aug 18, 2019, 10:11 PM Ralph Versteegen <teeemcee at gmail.com>
> wrote:
>
>> I decided to added usernames to the listing and summary table because
>> it's the easiest way to figure out which reports are coming from the same
>> person, especially if they only give their email address or name once. And
>> the username is typically visible in a lot of log output or .rpg path
>> anyway. But we could, for example, instead generate a unique ID on each
>> computer.
>>
>> On Mon, 19 Aug 2019 at 14:03, Ralph Versteegen <teeemcee at gmail.com>
>> wrote:
>>
>>> An update.
>>> http://tmc.castleparadox.com/ohr/reports20190819.txt
>>>
>>> We're currently up to 52 valid crash reports, and few empty or test
>>> ones. Here's a listing. There's better summary information now at the
>>> bottom of the file.
>>> There are a few interesting crashes, such as three identical ones inside
>>> in the Sell menu, with the comment "I was testing a game I was created, and
>>> the game crashed when I selected the sell menu. This also occurs in other
>>> games as well"
>>> Many people are hitting the MIDI loop crash, which causes quite a lot of
>>> different stacktraces.
>>>
>>> I'm still planning to create a page on the wiki with a table of the
>>> summaries where we can add additional notes and triage them, and links to
>>> the bug tracker (with detailed investigations of certain crashes). The wiki
>>> seems the easiest place for that, since we need to be able to edit. I need
>>> to write a script that can update it while preserving edits. But I don't
>>> want to put the more detailed information than summaries on the wiki, it
>>> would be too much. And what about privacy, should I even be putting
>>> usernames, .rpg filenames, partially-masked emails, which are in the
>>> summary table, in public view? What about the detailed version, which may
>>> include, for example, locales and filenames encountered while browsing?
>>>
>>> On Fri, 15 Mar 2019 at 16:39, Ralph Versteegen <teeemcee at gmail.com>
>>> wrote:
>>>
>>>> We've now collected a few crashes from CrashRpt. Currently showbug
>>>> doesn't generate a report, but I will add that too. There are also many
>>>> more errors that could be converted to showbug.
>>>> Even with just a 10 reports so far, it's a pain to go through them, so
>>>> I've written a tool, misc/process_crashreports.py, to analyse them.
>>>>
>>>> Generating a backtrace from the minidump was tricky -- I tried
>>>> Microsoft's windbg and cdb and CrashRpt's own crprober but they all
>>>> generated garbage, only Visual Studio's debugger (which is completely
>>>> separate) worked. So I used Breakpad to analyse the minidumps instead,
>>>> that's extra complexity but works great and is cross-platform and well
>>>> maintained (Breakpad is used by both Chrome and Firefox and has associated
>>>> client-side libraries to replace CrashRpt too not that I'm planning to)
>>>>
>>>> Unfortunately wine is still needed to generate the .sym files from the
>>>> .pdb file, so the process couldn't really be done on the server
>>>> automatically. We could easily generate the .sym files (eg during the
>>>> nightly build) and upload them with the pdbs, but still, if we can't run
>>>> git on the server, then we can't annotate the backtraces with source code.
>>>> But aside from that git isn't really needed, so I guess we could generate
>>>> and email nearly-full report summaries automatically with the crashrpt php
>>>> script.
>>>>
>>>> But otherwise, I'll just manually run the tool on the reports from time
>>>> to time.
>>>>
>>>> I've attached the generated summaries of those 10 reports.
>>>> There is a summary of the summaries at the bottom of the file.
>>>> Still not summarised enough. My summary of those summary-summaries is:
>>>> -one is a crash in DESCRIBE_TAG_AUTOSET_PLACES, which I know about but
>>>> have been lazy about fixing
>>>> -two are new to me, they are in MAPEDIT_APPEND_IMPORTED_TILEMAPS. I
>>>> found the cause of this. Hurrah! It was worth it!
>>>> -three are crashes while playing a MIDI file. So that still happens,
>>>> although I can't reproduce it anymore :(
>>>> -the last four are from Charbile's gfx_sdl2 testing
>>>> --one is inside SDL2.dll and is the crash he reported when
>>>> fullscreening at 1280*720
>>>> --the other three are in switch_gfx_backend, but this manifests as two
>>>> different backtraces
>>>>
>>>> We're going to need a better way to keep track of all of these. A text
>>>> file with annotations is probably good enough, plus filing bug reports for
>>>> crashes so we have a bug number to refer to.
>>>> But crashrpt actually has an accompanying web server/app for
>>>> collecting, summarising and filing reports:
>>>> http://crashfix.sourceforge.net/ CrashFix is probably quite
>>>> overwrought for this task though.
>>>> (To my surprise, it can actually parse minidumps and pdb files to in
>>>> order to produce stack traces, on linux. All other tools use Microsoft's
>>>> libraries to read pdb files, including Breakpad.)
>>>>
>>>> _______________________________________________
>> Ohrrpgce mailing list
>> ohrrpgce at lists.motherhamster.org
>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>
> _______________________________________________
> Ohrrpgce mailing list
> ohrrpgce at lists.motherhamster.org
> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20190820/40d52e2d/attachment-0001.html>