[Ohrrpgce] Importing bugzilla bugs into github

Ralph Versteegen teeemcee at gmail.com
Fri Mar 13 04:58:36 PDT 2020


Update! I have now completed making changes to and testing the
bugzilla2github tool. (I ended up making an awful lot.) I've imported all
the bugs into a test repository here:
https://github.com/ohrrpgce/testrepo/issues
This is the last chance to look over the results before I run it on the
real thing, which can't be undone.

I have replaced names and emails for certain people with their github
accounts:
email2login = {
    "__name__": "email to GitHub login",
    "teeemcee": "rversteegen",
    "caron.mike": "pkmnfrk",
    "Bob": "bob-the-hamster",
    "Bob-bugzilla": "bob-the-hamster",
    "mkidder": "fyrewulff",
    "neota": "0ion9",
    "sonichedgehog_hyperblast00": "MirceaKitsune",
    "sorlok_reaves": "sorlok",
    "ziggy": "ziggythehamster",
    "alkabanfall": "raekuul",
}
Is this a bad thing? Otherwise you would see both email prefix and name,
e.g. "Keith Gable <ziggy>". (Note that noone receives notifications from
the imported bugs.)
I see I missed @arperry.

I decided to squash "cosmetic" and "minor" severity levels into a "minor"
tag, and not convert the "major" and "critical" levels to tags, because I
found the application of those too dubious/spotty.

I have an "os: unix" label, and no label for linux.

There are bugs for OHRRPGCE FMF and Hamster Whisper (just the one).

Note about the testrepo: I was testing out interrupting and resuming the
run, which exposed a bug, and also made a change to the script halfway, so
bugs before 529 have broken cross-reference links, and also show "bz#300
(#300)" instead of just "bz#300".

I notice that links between duplicate bugs are one-way in the original xml
(unlike depends-on and blocked-by). Adding the reverse links to the bug
description might be the last change I make, since github doesn't generate
"mentioned by" cross-bug links if the mentioning bug hasn't been imported
yet.

On Tue, 25 Feb 2020 at 23:25, Ralph Versteegen <teeemcee at gmail.com> wrote:

> Projects just seem like a handy way to maintain a todo list and an
> *indication* of goal for the next release without needing to actually file
> a bunch of feature requests in the bug tracker. The Plans on the wiki tend
> to be very long term and out of date. I would definitely continue to use
> the wiki (and emails) for detailed plans, not the bug tracker.
> My actual todo list is completely disorganised, unprioritised, too long,
> and often doesn't actually say what my todos for the next release are, so I
> don't even refer to it myself! A public todo list (people sometimes ask me)
> would be nice to have.
>
> Come to think of it, there's no real point putting projects for OHRRPGCE
> releases in the ohrrpgce organisation, because releases never depend on
> anything that might be in another repo (or in web or tools). And even if we
> had separate web/tools github repos, we may as well just keep issues for
> HWhisper, etc, all in the oe bug tracker. (There were bugs for HW and even
> Seth's OHR FMF on bugzilla anyway.)
>
> Splitting the git repos won't require any changes to svn. In fact, the web
> and tools branches could continue to be stored in the same git repo on the
> svn-git cron machine, and just get uploaded to separate github repos. And
> it's already possible for someone to clone the bitbucket/github repo three
> times locally, downloading just web or tools branches, or everything else.
> Complex to explain that to someone, though.
>
> On Mon, 24 Feb 2020 at 23:49, James Paige <Bob at hamsterrepublic.com> wrote:
>
>> I have no strong preference for whether we use projects or milestones. I
>> guess for bigger projects with more people there must be a use case for
>> both.
>>
>> Splitting web and tools into different git repos sounds good to me. If
>> that breaks anything regarding the subversion repo, I would be okay with
>> splitting them out of the svn repo also if needed.
>>
>> On Sun, Feb 23, 2020, 8:16 PM Ralph Versteegen <teeemcee at gmail.com>
>> wrote:
>>
>>> An update: I haven't worked on this for a couple weeks, but I was pretty
>>> close to having it working, just making tweaks. Want to get it done before
>>> I get distracted with this year's 7DRL. All the old attachments are
>>> uploaded (along with http://rpg.hamsterrepublic.com/bugzilla/quips.txt
>>> and http://rpg.hamsterrepublic.com/bugzilla/buglist.xml). Then I can
>>> look at Sourceforge.
>>>
>>> I'd never paid much attention to "Projects" on Github before, but they
>>> seem more useful than using Milestones for todo lists. Milestones let you
>>> reorder issues/PRs, but Projects let you also sort into columns, eg for
>>> priority. The project UI is very different. But the most important feature
>>> is that you can add note cards, not just issues. They can also contain
>>> issues from multiple repositories, although project-issue crosslinking only
>>> works if project and issue are in the same repo or it's a project in the
>>> 'ohrrpgce' organisation and the issue is in a repo under the ohrrpgce
>>> organisation**.
>>>
>>> Compare:
>>> https://github.com/ohrrpgce/ohrrpgce/milestone/1
>>> https://github.com/orgs/ohrrpgce/projects/1
>>>
>>> But milestones and projects seem pretty redundant to each other. I
>>> wonder whether to use milestones at all. (Maybe just to tag closed bugs,
>>> not as a todo list for a release?)
>>>
>>> ** Adding an issue to a project crosslinks it from the issue itself, see
>>> eg https://github.com/ohrrpgce/ohrrpgce/issues/5.
>>> We might want to add more repos to the ohrrpgce organisation, so could
>>> make sense to put the projects there instead of the repo. For example, I'm
>>> thinking of splitting 'web' and 'tools' branches out to separate git repos,
>>> since it's pretty inconvenient and strange to have them as orphaned
>>> branches.
>>> I'm also wondering whether a separate repo where crash reports are
>>> automatically uploaded to could be a handy way to manage them.
>>>
>>>
>>> On Sat, 8 Feb 2020 at 01:51, James Paige <Bob at hamsterrepublic.com>
>>> wrote:
>>>
>>>> That is even better!
>>>>
>>>> On Fri, Feb 7, 2020, 4:50 AM Ralph Versteegen <teeemcee at gmail.com>
>>>> wrote:
>>>>
>>>>> https://rpg.hamsterrepublic.com/bugzilla/ already exists (it's a
>>>>> redirect to the sf buglist, plus hosts
>>>>> https://rpg.hamsterrepublic.com/bugzilla/buglist.txt). So I guess the
>>>>> logical URL to use is
>>>>> https://rpg.hamsterrepublic.com/bugzilla/old-attachments/
>>>>>
>>>>> <https://rpg.hamsterrepublic.com/old-bugzilla-attachments/>
>>>>>
>>>>> On Fri, 7 Feb 2020 at 14:40, James Paige <Bob at hamsterrepublic.com>
>>>>> wrote:
>>>>>
>>>>>> Oops! Yes!
>>>>>>
>>>>>> How about putting them at
>>>>>> https://rpg.hamsterrepublic.com/old-bugzilla-attachments/ ?
>>>>>>
>>>>>>
>>>>>> On Thu, Feb 6, 2020, 5:53 PM Ralph Versteegen <teeemcee at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Oh, right! The main reason I wrote that email was to ask about
>>>>>>> uploading the bug attachments somewhere. I see there are still various
>>>>>>> redirects and remnants of bugzilla left behind.
>>>>>>>
>>>>>>> On Fri, 7 Feb 2020 at 02:47, Ralph Versteegen <teeemcee at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I agree that too many labels aren't helpful. Labels are most useful
>>>>>>>> when you can't just do a text search (all crash bugs will be easily found
>>>>>>>> with a search for "crash"), eg for bug priorities.
>>>>>>>> I think maybe I'll combine "cosmetic" and "minor" into a single
>>>>>>>> "minor" tag, since we likely want that one anyway while "cosmetic" is too
>>>>>>>> specific. Wait, how much difference is there between "minor" and "needs
>>>>>>>> improvement"?  Plenty of outright bugs are merely cosmetic. Argh!
>>>>>>>> Really, "bug" is a region of the continuum (nearly the whole
>>>>>>>> continuum) from feature request to "needs improvement" to blocker.
>>>>>>>>
>>>>>>>> But... major/high priority bugs is possible the most likely thing
>>>>>>>> of all that we would want to categorise and search for. And they aren't
>>>>>>>> necessarily going to be put on the todo list for the next release
>>>>>>>> (milestone). So I guess I should map major/critical/blocker to "major".
>>>>>>>> Although the categorisation of bug severities on bugzilla is pretty
>>>>>>>> dubious, it is a useful starting point.
>>>>>>>>
>>>>>>>> 7d344553-34f0-0310-a9b1-970ce8f1c3a2 is the "Repository UUID" you
>>>>>>>> see when running "svn info".
>>>>>>>> It's also possible to give git-svn a file with mapping from
>>>>>>>> username to actual email, but that won't apply retroactively.
>>>>>>>>
>>>>>>>> On Fri, 7 Feb 2020 at 02:08, James Paige <Bob at hamsterrepublic.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Cool!
>>>>>>>>>
>>>>>>>>> I like the idea of rel: and resolved: prefixes.
>>>>>>>>>
>>>>>>>>> I vote for eliminating the "bug" tag completely. In general I say
>>>>>>>>> err on the side of fewer tags when the specific need for a tag is not
>>>>>>>>> obvious.
>>>>>>>>>
>>>>>>>>> That GitHub contributions trick is cool. I'll try it. Is
>>>>>>>>> 7d344553-34f0-0310-a9b1-970ce8f1c3a2 the internal ID of our svn
>>>>>>>>> repo?
>>>>>>>>>
>>>>>>>>> On Thu, Feb 6, 2020, 8:04 AM Ralph Versteegen <teeemcee at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, 7 Feb 2020 at 01:43, Ralph Versteegen <teeemcee at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I've been putting this off for years (two years since my last
>>>>>>>>>>> attempt in fact) but I've finally managed to get bugzilla running locally
>>>>>>>>>>> and import the databases so that I can export the buglist as xml and use this
>>>>>>>>>>> script
>>>>>>>>>>> <https://gist.github.com/Zimmi48/d923e52f64fe17c72852d9c148bfcdc6#file-bugzilla2github>
>>>>>>>>>>> to import it into github.
>>>>>>>>>>>
>>>>>>>>>>> I'm now in the process of customising the script and the mapping
>>>>>>>>>>> to labels.
>>>>>>>>>>>
>>>>>>>>>>> Github doesn't have an API that allows attaching files to issue
>>>>>>>>>>> comments, so I suggest that we upload all the attachments to a directory on
>>>>>>>>>>> hamsterrepublic.com so that we can link to them.
>>>>>>>>>>> (Unfortunately I first have to write a script to extract them
>>>>>>>>>>> out of the bugzilla DB.)
>>>>>>>>>>>
>>>>>>>>>>> With all the labels for components, versions, severity, status,
>>>>>>>>>>> etc, we'll have a lot of labels. I think it makes sense to organise our
>>>>>>>>>>> labels a bit better. All version labels could be prefixed with "rel: ",
>>>>>>>>>>> like "rel: alectormancy". And maybe duplicate, wontfix, invalid,
>>>>>>>>>>> worksforme, tooconfusing
>>>>>>>>>>> could be prefixed with "resolved:" or "closed:"?
>>>>>>>>>>>
>>>>>>>>>>> I'm going to translate the "feature request", "cosmetic",
>>>>>>>>>>> "blocker" severities into labels. I think I'll leave behind "major",
>>>>>>>>>>> "minor", "normal" and "critical", and the priority levels. They (if not
>>>>>>>>>>> default value) will still be in the bug description.
>>>>>>>>>>>
>>>>>>>>>>> There isn't really anything that corresponds closely to our gh
>>>>>>>>>>> "bug" and "needs improvement" labels. I wonder whether we should just
>>>>>>>>>>> remove the "bug" label, since although it's informative and useful in
>>>>>>>>>>> searches, it's already a nuisance to accurately tag all the bugs** and
>>>>>>>>>>> we'll get an extra 1000 bugs missing it. "bug" is basically the absense of
>>>>>>>>>>> numerous special categories like "new feature", "invalid" and "tracking".
>>>>>>>>>>>
>>>>>>>>>> Or, I could just apply "bug" as a guess according to something
>>>>>>>>>> like that simple rule, and we can fix up any mistakes. Will be looking
>>>>>>>>>> through all the open imported bugs anyway.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> **See
>>>>>>>>>>> https://github.com/ohrrpgce/ohrrpgce/issues?utf8=%E2%9C%93&q=-label%3Abug+-label%3A%22needs+improvement%22++-label%3A%22new+feature%22
>>>>>>>>>>>
>>>>>>>>>>> Bugzilla hides people's email addresses (by stripping the
>>>>>>>>>>> domain) by default so user names will be something like "David Gowers
>>>>>>>>>>> <00ai99>". I think that's fine. I can (am planning to) also replace the
>>>>>>>>>>> username with a github @ mention (it shouldn't cause any notifications,
>>>>>>>>>>> because this is the mass import API), but the only people who I know the
>>>>>>>>>>> github username of are myself, James and Mike. Doesn't matter.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> BTW, I found that it's possible to make svn commits copied to
>>>>>>>>>>> the git-svn mirror on github count towards your "contributions" shown on
>>>>>>>>>>> your github profile page. To do so, add a new email address to your github
>>>>>>>>>>> account with the form svncommitername at 7d344553-34f0-0310-a9b1-970ce8f1c3a2,
>>>>>>>>>>> where svncommitername is e.g. teeemcee or james. The email can be private.
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>
>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>
>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>> Ohrrpgce mailing list
>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>
>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Ohrrpgce mailing list
>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>
>>>>> _______________________________________________
>>>>> Ohrrpgce mailing list
>>>>> ohrrpgce at lists.motherhamster.org
>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>
>>>> _______________________________________________
>>>> Ohrrpgce mailing list
>>>> ohrrpgce at lists.motherhamster.org
>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>
>>> _______________________________________________
>>> Ohrrpgce mailing list
>>> ohrrpgce at lists.motherhamster.org
>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>
>> _______________________________________________
>> Ohrrpgce mailing list
>> ohrrpgce at lists.motherhamster.org
>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20200314/67899e1e/attachment-0001.html>


More information about the Ohrrpgce mailing list