[Ohrrpgce] Importing bugzilla bugs into github

James Paige Bob at hamsterrepublic.com
Sat Mar 14 15:21:09 PDT 2020 

I disabled e-mails for anything other than new ticket creation on the
sourceforge bug tracker. I also added text that links tot he github issue
tracker

I haven't figure out how to disable the sourceforge bug tracker completely,
but that is what I want to do.

On Sat, Mar 14, 2020 at 12:42 PM Ralph Versteegen <teeemcee at gmail.com>
wrote:

> It's done! Bugzilla and Sourceforge are migrated.
> I've updated the wiki, posted a News item, set up a redirect from
> bugzilla/show_bug.cgi?id=X (which translates both bugzilla and sourceforge
> bug ids), and deleted the testrepo.
>
> Now we have to actually sort through the 297 open issues :S
>
> James, if you want us to post redirects on the existing SF bugs, maybe you
> could you (temporarily) disable the mailing list email notifications? If we
> don't lock that buglist then  we'll want to turn them back on afterwards.
>
> On Sat, 14 Mar 2020 at 18:06, Ralph Versteegen <teeemcee at gmail.com> wrote:
>
>> I've adapted the script (which I will also upload to GH, but it's a fork
>> of a gist instead of a fork of the actual bugzilla2github repo; the history
>> needs cleaning) to also import from SourceForge to GitHub. I didn't like
>> the perl script I found to do the import, it wasn't flexible, I don't know
>> perl, and this way we get the same formatting for bugs from both sources.
>> That was pretty easy actually!
>>
>> Bugs are in the test repo: https://github.com/ohrrpgce/testrepo/issues?q=
>>
>> I made a change to the formatting: the original bug number is now at the
>> top of the description so that it shows in the mouse hover preview.
>>
>> I added @arperry @Spencer-Zhang and @kveroneau to the usernames list.
>>
>> I downloaded the (4) bug attachments from SF and uploaded them to the
>> same place as the bugzilla ones, so we're not reliant on SF at all.
>>
>> I've updated the http://rpg.hamsterrepublic.com/bugzilla redirect to
>> point to GH. I think there were other redirects?
>> After the import I'll try make it redirect from to the original bugzilla
>> bug to the imported one, which should be easy because I will have the
>> lookup table for it.
>>
>> On Sat, 14 Mar 2020 at 01:19, James Paige <Bob at hamsterrepublic.com>
>> wrote:
>>
>>> This looks fantastic! Very nice.
>>>
>>> I think the name mapping is a good idea.
>>>
>>> I see I reason not to proceed
>>>
>>> On Fri, Mar 13, 2020, 7:58 AM Ralph Versteegen <teeemcee at gmail.com>
>>> wrote:
>>>
>>>> Update! I have now completed making changes to and testing the
>>>> bugzilla2github tool. (I ended up making an awful lot.) I've imported all
>>>> the bugs into a test repository here:
>>>> https://github.com/ohrrpgce/testrepo/issues
>>>> This is the last chance to look over the results before I run it on the
>>>> real thing, which can't be undone.
>>>>
>>>> I have replaced names and emails for certain people with their github
>>>> accounts:
>>>> email2login = {
>>>>     "__name__": "email to GitHub login",
>>>>     "teeemcee": "rversteegen",
>>>>     "caron.mike": "pkmnfrk",
>>>>     "Bob": "bob-the-hamster",
>>>>     "Bob-bugzilla": "bob-the-hamster",
>>>>     "mkidder": "fyrewulff",
>>>>     "neota": "0ion9",
>>>>     "sonichedgehog_hyperblast00": "MirceaKitsune",
>>>>     "sorlok_reaves": "sorlok",
>>>>     "ziggy": "ziggythehamster",
>>>>     "alkabanfall": "raekuul",
>>>> }
>>>> Is this a bad thing? Otherwise you would see both email prefix and
>>>> name, e.g. "Keith Gable <ziggy>". (Note that noone receives notifications
>>>> from the imported bugs.)
>>>> I see I missed @arperry.
>>>>
>>>> I decided to squash "cosmetic" and "minor" severity levels into a
>>>> "minor" tag, and not convert the "major" and "critical" levels to tags,
>>>> because I found the application of those too dubious/spotty.
>>>>
>>>> I have an "os: unix" label, and no label for linux.
>>>>
>>>> There are bugs for OHRRPGCE FMF and Hamster Whisper (just the one).
>>>>
>>>> Note about the testrepo: I was testing out interrupting and resuming
>>>> the run, which exposed a bug, and also made a change to the script halfway,
>>>> so bugs before 529 have broken cross-reference links, and also show "bz#300
>>>> (#300)" instead of just "bz#300".
>>>>
>>>> I notice that links between duplicate bugs are one-way in the original
>>>> xml (unlike depends-on and blocked-by). Adding the reverse links to the bug
>>>> description might be the last change I make, since github doesn't generate
>>>> "mentioned by" cross-bug links if the mentioning bug hasn't been imported
>>>> yet.
>>>>
>>>> On Tue, 25 Feb 2020 at 23:25, Ralph Versteegen <teeemcee at gmail.com>
>>>> wrote:
>>>>
>>>>> Projects just seem like a handy way to maintain a todo list and an
>>>>> *indication* of goal for the next release without needing to actually file
>>>>> a bunch of feature requests in the bug tracker. The Plans on the wiki tend
>>>>> to be very long term and out of date. I would definitely continue to use
>>>>> the wiki (and emails) for detailed plans, not the bug tracker.
>>>>> My actual todo list is completely disorganised, unprioritised, too
>>>>> long, and often doesn't actually say what my todos for the next release
>>>>> are, so I don't even refer to it myself! A public todo list (people
>>>>> sometimes ask me) would be nice to have.
>>>>>
>>>>> Come to think of it, there's no real point putting projects for
>>>>> OHRRPGCE releases in the ohrrpgce organisation, because releases never
>>>>> depend on anything that might be in another repo (or in web or tools). And
>>>>> even if we had separate web/tools github repos, we may as well just keep
>>>>> issues for HWhisper, etc, all in the oe bug tracker. (There were bugs for
>>>>> HW and even Seth's OHR FMF on bugzilla anyway.)
>>>>>
>>>>> Splitting the git repos won't require any changes to svn. In fact, the
>>>>> web and tools branches could continue to be stored in the same git repo on
>>>>> the svn-git cron machine, and just get uploaded to separate github repos.
>>>>> And it's already possible for someone to clone the bitbucket/github repo
>>>>> three times locally, downloading just web or tools branches, or everything
>>>>> else. Complex to explain that to someone, though.
>>>>>
>>>>> On Mon, 24 Feb 2020 at 23:49, James Paige <Bob at hamsterrepublic.com>
>>>>> wrote:
>>>>>
>>>>>> I have no strong preference for whether we use projects or
>>>>>> milestones. I guess for bigger projects with more people there must be a
>>>>>> use case for both.
>>>>>>
>>>>>> Splitting web and tools into different git repos sounds good to me.
>>>>>> If that breaks anything regarding the subversion repo, I would be okay with
>>>>>> splitting them out of the svn repo also if needed.
>>>>>>
>>>>>> On Sun, Feb 23, 2020, 8:16 PM Ralph Versteegen <teeemcee at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> An update: I haven't worked on this for a couple weeks, but I was
>>>>>>> pretty close to having it working, just making tweaks. Want to get it done
>>>>>>> before I get distracted with this year's 7DRL. All the old attachments are
>>>>>>> uploaded (along with
>>>>>>> http://rpg.hamsterrepublic.com/bugzilla/quips.txt and
>>>>>>> http://rpg.hamsterrepublic.com/bugzilla/buglist.xml). Then I can
>>>>>>> look at Sourceforge.
>>>>>>>
>>>>>>> I'd never paid much attention to "Projects" on Github before, but
>>>>>>> they seem more useful than using Milestones for todo lists. Milestones let
>>>>>>> you reorder issues/PRs, but Projects let you also sort into columns, eg for
>>>>>>> priority. The project UI is very different. But the most important feature
>>>>>>> is that you can add note cards, not just issues. They can also contain
>>>>>>> issues from multiple repositories, although project-issue crosslinking only
>>>>>>> works if project and issue are in the same repo or it's a project in the
>>>>>>> 'ohrrpgce' organisation and the issue is in a repo under the ohrrpgce
>>>>>>> organisation**.
>>>>>>>
>>>>>>> Compare:
>>>>>>> https://github.com/ohrrpgce/ohrrpgce/milestone/1
>>>>>>> https://github.com/orgs/ohrrpgce/projects/1
>>>>>>>
>>>>>>> But milestones and projects seem pretty redundant to each other. I
>>>>>>> wonder whether to use milestones at all. (Maybe just to tag closed bugs,
>>>>>>> not as a todo list for a release?)
>>>>>>>
>>>>>>> ** Adding an issue to a project crosslinks it from the issue itself,
>>>>>>> see eg https://github.com/ohrrpgce/ohrrpgce/issues/5.
>>>>>>> We might want to add more repos to the ohrrpgce organisation, so
>>>>>>> could make sense to put the projects there instead of the repo. For
>>>>>>> example, I'm thinking of splitting 'web' and 'tools' branches out to
>>>>>>> separate git repos, since it's pretty inconvenient and strange to have them
>>>>>>> as orphaned branches.
>>>>>>> I'm also wondering whether a separate repo where crash reports are
>>>>>>> automatically uploaded to could be a handy way to manage them.
>>>>>>>
>>>>>>>
>>>>>>> On Sat, 8 Feb 2020 at 01:51, James Paige <Bob at hamsterrepublic.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> That is even better!
>>>>>>>>
>>>>>>>> On Fri, Feb 7, 2020, 4:50 AM Ralph Versteegen <teeemcee at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> https://rpg.hamsterrepublic.com/bugzilla/ already exists (it's a
>>>>>>>>> redirect to the sf buglist, plus hosts
>>>>>>>>> https://rpg.hamsterrepublic.com/bugzilla/buglist.txt). So I guess
>>>>>>>>> the logical URL to use is
>>>>>>>>> https://rpg.hamsterrepublic.com/bugzilla/old-attachments/
>>>>>>>>>
>>>>>>>>> <https://rpg.hamsterrepublic.com/old-bugzilla-attachments/>
>>>>>>>>>
>>>>>>>>> On Fri, 7 Feb 2020 at 14:40, James Paige <Bob at hamsterrepublic.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Oops! Yes!
>>>>>>>>>>
>>>>>>>>>> How about putting them at
>>>>>>>>>> https://rpg.hamsterrepublic.com/old-bugzilla-attachments/ ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Feb 6, 2020, 5:53 PM Ralph Versteegen <teeemcee at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Oh, right! The main reason I wrote that email was to ask about
>>>>>>>>>>> uploading the bug attachments somewhere. I see there are still various
>>>>>>>>>>> redirects and remnants of bugzilla left behind.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, 7 Feb 2020 at 02:47, Ralph Versteegen <
>>>>>>>>>>> teeemcee at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I agree that too many labels aren't helpful. Labels are most
>>>>>>>>>>>> useful when you can't just do a text search (all crash bugs will be easily
>>>>>>>>>>>> found with a search for "crash"), eg for bug priorities.
>>>>>>>>>>>> I think maybe I'll combine "cosmetic" and "minor" into a single
>>>>>>>>>>>> "minor" tag, since we likely want that one anyway while "cosmetic" is too
>>>>>>>>>>>> specific. Wait, how much difference is there between "minor" and "needs
>>>>>>>>>>>> improvement"?  Plenty of outright bugs are merely cosmetic. Argh!
>>>>>>>>>>>> Really, "bug" is a region of the continuum (nearly the whole
>>>>>>>>>>>> continuum) from feature request to "needs improvement" to blocker.
>>>>>>>>>>>>
>>>>>>>>>>>> But... major/high priority bugs is possible the most likely
>>>>>>>>>>>> thing of all that we would want to categorise and search for. And they
>>>>>>>>>>>> aren't necessarily going to be put on the todo list for the next release
>>>>>>>>>>>> (milestone). So I guess I should map major/critical/blocker to "major".
>>>>>>>>>>>> Although the categorisation of bug severities on bugzilla is pretty
>>>>>>>>>>>> dubious, it is a useful starting point.
>>>>>>>>>>>>
>>>>>>>>>>>> 7d344553-34f0-0310-a9b1-970ce8f1c3a2 is the "Repository UUID"
>>>>>>>>>>>> you see when running "svn info".
>>>>>>>>>>>> It's also possible to give git-svn a file with mapping from
>>>>>>>>>>>> username to actual email, but that won't apply retroactively.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, 7 Feb 2020 at 02:08, James Paige <
>>>>>>>>>>>> Bob at hamsterrepublic.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Cool!
>>>>>>>>>>>>>
>>>>>>>>>>>>> I like the idea of rel: and resolved: prefixes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I vote for eliminating the "bug" tag completely. In general I
>>>>>>>>>>>>> say err on the side of fewer tags when the specific need for a tag is not
>>>>>>>>>>>>> obvious.
>>>>>>>>>>>>>
>>>>>>>>>>>>> That GitHub contributions trick is cool. I'll try it. Is
>>>>>>>>>>>>> 7d344553-34f0-0310-a9b1-970ce8f1c3a2 the internal ID of our
>>>>>>>>>>>>> svn repo?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Feb 6, 2020, 8:04 AM Ralph Versteegen <
>>>>>>>>>>>>> teeemcee at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, 7 Feb 2020 at 01:43, Ralph Versteegen <
>>>>>>>>>>>>>> teeemcee at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've been putting this off for years (two years since my
>>>>>>>>>>>>>>> last attempt in fact) but I've finally managed to get bugzilla running
>>>>>>>>>>>>>>> locally and import the databases so that I can export the buglist as xml
>>>>>>>>>>>>>>> and use this script
>>>>>>>>>>>>>>> <https://gist.github.com/Zimmi48/d923e52f64fe17c72852d9c148bfcdc6#file-bugzilla2github>
>>>>>>>>>>>>>>> to import it into github.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm now in the process of customising the script and the
>>>>>>>>>>>>>>> mapping to labels.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Github doesn't have an API that allows attaching files to
>>>>>>>>>>>>>>> issue comments, so I suggest that we upload all the attachments to a
>>>>>>>>>>>>>>> directory on hamsterrepublic.com so that we can link to
>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>> (Unfortunately I first have to write a script to extract
>>>>>>>>>>>>>>> them out of the bugzilla DB.)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> With all the labels for components, versions, severity,
>>>>>>>>>>>>>>> status, etc, we'll have a lot of labels. I think it makes sense to organise
>>>>>>>>>>>>>>> our labels a bit better. All version labels could be prefixed with "rel: ",
>>>>>>>>>>>>>>> like "rel: alectormancy". And maybe duplicate, wontfix, invalid,
>>>>>>>>>>>>>>> worksforme, tooconfusing
>>>>>>>>>>>>>>> could be prefixed with "resolved:" or "closed:"?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm going to translate the "feature request", "cosmetic",
>>>>>>>>>>>>>>> "blocker" severities into labels. I think I'll leave behind "major",
>>>>>>>>>>>>>>> "minor", "normal" and "critical", and the priority levels. They (if not
>>>>>>>>>>>>>>> default value) will still be in the bug description.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There isn't really anything that corresponds closely to our
>>>>>>>>>>>>>>> gh "bug" and "needs improvement" labels. I wonder whether we should just
>>>>>>>>>>>>>>> remove the "bug" label, since although it's informative and useful in
>>>>>>>>>>>>>>> searches, it's already a nuisance to accurately tag all the bugs** and
>>>>>>>>>>>>>>> we'll get an extra 1000 bugs missing it. "bug" is basically the absense of
>>>>>>>>>>>>>>> numerous special categories like "new feature", "invalid" and "tracking".
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Or, I could just apply "bug" as a guess according to
>>>>>>>>>>>>>> something like that simple rule, and we can fix up any mistakes. Will be
>>>>>>>>>>>>>> looking through all the open imported bugs anyway.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> **See
>>>>>>>>>>>>>>> https://github.com/ohrrpgce/ohrrpgce/issues?utf8=%E2%9C%93&q=-label%3Abug+-label%3A%22needs+improvement%22++-label%3A%22new+feature%22
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Bugzilla hides people's email addresses (by stripping the
>>>>>>>>>>>>>>> domain) by default so user names will be something like "David Gowers
>>>>>>>>>>>>>>> <00ai99>". I think that's fine. I can (am planning to) also replace the
>>>>>>>>>>>>>>> username with a github @ mention (it shouldn't cause any notifications,
>>>>>>>>>>>>>>> because this is the mass import API), but the only people who I know the
>>>>>>>>>>>>>>> github username of are myself, James and Mike. Doesn't matter.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> BTW, I found that it's possible to make svn commits copied
>>>>>>>>>>>>>>> to the git-svn mirror on github count towards your "contributions" shown on
>>>>>>>>>>>>>>> your github profile page. To do so, add a new email address to your github
>>>>>>>>>>>>>>> account with the form svncommitername at 7d344553-34f0-0310-a9b1-970ce8f1c3a2,
>>>>>>>>>>>>>>> where svncommitername is e.g. teeemcee or james. The email can be private.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>>>>
>>>>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>>
>>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>
>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>
>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Ohrrpgce mailing list
>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>
>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ohrrpgce mailing list
>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>
>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Ohrrpgce mailing list
>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>
>>>>> _______________________________________________
>>>> Ohrrpgce mailing list
>>>> ohrrpgce at lists.motherhamster.org
>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>
>>> _______________________________________________
>>> Ohrrpgce mailing list
>>> ohrrpgce at lists.motherhamster.org
>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>
>> _______________________________________________
> Ohrrpgce mailing list
> ohrrpgce at lists.motherhamster.org
> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20200314/1b929161/attachment-0001.html>

More information about the Ohrrpgce mailing list