[Ohrrpgce] Importing bugzilla bugs into github

James Paige Bob at hamsterrepublic.com
Sat Mar 14 15:24:14 PDT 2020 

Okay, I think I figured out how to block new ticket creation on sourceforge

On Sat, Mar 14, 2020 at 6:21 PM James Paige <Bob at hamsterrepublic.com> wrote:

> I disabled e-mails for anything other than new ticket creation on the
> sourceforge bug tracker. I also added text that links tot he github issue
> tracker
>
> I haven't figure out how to disable the sourceforge bug tracker
> completely, but that is what I want to do.
>
> On Sat, Mar 14, 2020 at 12:42 PM Ralph Versteegen <teeemcee at gmail.com>
> wrote:
>
>> It's done! Bugzilla and Sourceforge are migrated.
>> I've updated the wiki, posted a News item, set up a redirect from
>> bugzilla/show_bug.cgi?id=X (which translates both bugzilla and sourceforge
>> bug ids), and deleted the testrepo.
>>
>> Now we have to actually sort through the 297 open issues :S
>>
>> James, if you want us to post redirects on the existing SF bugs, maybe
>> you could you (temporarily) disable the mailing list email notifications?
>> If we don't lock that buglist then  we'll want to turn them back on
>> afterwards.
>>
>> On Sat, 14 Mar 2020 at 18:06, Ralph Versteegen <teeemcee at gmail.com>
>> wrote:
>>
>>> I've adapted the script (which I will also upload to GH, but it's a fork
>>> of a gist instead of a fork of the actual bugzilla2github repo; the history
>>> needs cleaning) to also import from SourceForge to GitHub. I didn't like
>>> the perl script I found to do the import, it wasn't flexible, I don't know
>>> perl, and this way we get the same formatting for bugs from both sources.
>>> That was pretty easy actually!
>>>
>>> Bugs are in the test repo:
>>> https://github.com/ohrrpgce/testrepo/issues?q=
>>>
>>> I made a change to the formatting: the original bug number is now at the
>>> top of the description so that it shows in the mouse hover preview.
>>>
>>> I added @arperry @Spencer-Zhang and @kveroneau to the usernames list.
>>>
>>> I downloaded the (4) bug attachments from SF and uploaded them to the
>>> same place as the bugzilla ones, so we're not reliant on SF at all.
>>>
>>> I've updated the http://rpg.hamsterrepublic.com/bugzilla redirect to
>>> point to GH. I think there were other redirects?
>>> After the import I'll try make it redirect from to the original bugzilla
>>> bug to the imported one, which should be easy because I will have the
>>> lookup table for it.
>>>
>>> On Sat, 14 Mar 2020 at 01:19, James Paige <Bob at hamsterrepublic.com>
>>> wrote:
>>>
>>>> This looks fantastic! Very nice.
>>>>
>>>> I think the name mapping is a good idea.
>>>>
>>>> I see I reason not to proceed
>>>>
>>>> On Fri, Mar 13, 2020, 7:58 AM Ralph Versteegen <teeemcee at gmail.com>
>>>> wrote:
>>>>
>>>>> Update! I have now completed making changes to and testing the
>>>>> bugzilla2github tool. (I ended up making an awful lot.) I've imported all
>>>>> the bugs into a test repository here:
>>>>> https://github.com/ohrrpgce/testrepo/issues
>>>>> This is the last chance to look over the results before I run it on
>>>>> the real thing, which can't be undone.
>>>>>
>>>>> I have replaced names and emails for certain people with their github
>>>>> accounts:
>>>>> email2login = {
>>>>>     "__name__": "email to GitHub login",
>>>>>     "teeemcee": "rversteegen",
>>>>>     "caron.mike": "pkmnfrk",
>>>>>     "Bob": "bob-the-hamster",
>>>>>     "Bob-bugzilla": "bob-the-hamster",
>>>>>     "mkidder": "fyrewulff",
>>>>>     "neota": "0ion9",
>>>>>     "sonichedgehog_hyperblast00": "MirceaKitsune",
>>>>>     "sorlok_reaves": "sorlok",
>>>>>     "ziggy": "ziggythehamster",
>>>>>     "alkabanfall": "raekuul",
>>>>> }
>>>>> Is this a bad thing? Otherwise you would see both email prefix and
>>>>> name, e.g. "Keith Gable <ziggy>". (Note that noone receives notifications
>>>>> from the imported bugs.)
>>>>> I see I missed @arperry.
>>>>>
>>>>> I decided to squash "cosmetic" and "minor" severity levels into a
>>>>> "minor" tag, and not convert the "major" and "critical" levels to tags,
>>>>> because I found the application of those too dubious/spotty.
>>>>>
>>>>> I have an "os: unix" label, and no label for linux.
>>>>>
>>>>> There are bugs for OHRRPGCE FMF and Hamster Whisper (just the one).
>>>>>
>>>>> Note about the testrepo: I was testing out interrupting and resuming
>>>>> the run, which exposed a bug, and also made a change to the script halfway,
>>>>> so bugs before 529 have broken cross-reference links, and also show "bz#300
>>>>> (#300)" instead of just "bz#300".
>>>>>
>>>>> I notice that links between duplicate bugs are one-way in the original
>>>>> xml (unlike depends-on and blocked-by). Adding the reverse links to the bug
>>>>> description might be the last change I make, since github doesn't generate
>>>>> "mentioned by" cross-bug links if the mentioning bug hasn't been imported
>>>>> yet.
>>>>>
>>>>> On Tue, 25 Feb 2020 at 23:25, Ralph Versteegen <teeemcee at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Projects just seem like a handy way to maintain a todo list and an
>>>>>> *indication* of goal for the next release without needing to actually file
>>>>>> a bunch of feature requests in the bug tracker. The Plans on the wiki tend
>>>>>> to be very long term and out of date. I would definitely continue to use
>>>>>> the wiki (and emails) for detailed plans, not the bug tracker.
>>>>>> My actual todo list is completely disorganised, unprioritised, too
>>>>>> long, and often doesn't actually say what my todos for the next release
>>>>>> are, so I don't even refer to it myself! A public todo list (people
>>>>>> sometimes ask me) would be nice to have.
>>>>>>
>>>>>> Come to think of it, there's no real point putting projects for
>>>>>> OHRRPGCE releases in the ohrrpgce organisation, because releases never
>>>>>> depend on anything that might be in another repo (or in web or tools). And
>>>>>> even if we had separate web/tools github repos, we may as well just keep
>>>>>> issues for HWhisper, etc, all in the oe bug tracker. (There were bugs for
>>>>>> HW and even Seth's OHR FMF on bugzilla anyway.)
>>>>>>
>>>>>> Splitting the git repos won't require any changes to svn. In fact,
>>>>>> the web and tools branches could continue to be stored in the same git repo
>>>>>> on the svn-git cron machine, and just get uploaded to separate github
>>>>>> repos. And it's already possible for someone to clone the bitbucket/github
>>>>>> repo three times locally, downloading just web or tools branches, or
>>>>>> everything else. Complex to explain that to someone, though.
>>>>>>
>>>>>> On Mon, 24 Feb 2020 at 23:49, James Paige <Bob at hamsterrepublic.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I have no strong preference for whether we use projects or
>>>>>>> milestones. I guess for bigger projects with more people there must be a
>>>>>>> use case for both.
>>>>>>>
>>>>>>> Splitting web and tools into different git repos sounds good to me.
>>>>>>> If that breaks anything regarding the subversion repo, I would be okay with
>>>>>>> splitting them out of the svn repo also if needed.
>>>>>>>
>>>>>>> On Sun, Feb 23, 2020, 8:16 PM Ralph Versteegen <teeemcee at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> An update: I haven't worked on this for a couple weeks, but I was
>>>>>>>> pretty close to having it working, just making tweaks. Want to get it done
>>>>>>>> before I get distracted with this year's 7DRL. All the old attachments are
>>>>>>>> uploaded (along with
>>>>>>>> http://rpg.hamsterrepublic.com/bugzilla/quips.txt and
>>>>>>>> http://rpg.hamsterrepublic.com/bugzilla/buglist.xml). Then I can
>>>>>>>> look at Sourceforge.
>>>>>>>>
>>>>>>>> I'd never paid much attention to "Projects" on Github before, but
>>>>>>>> they seem more useful than using Milestones for todo lists. Milestones let
>>>>>>>> you reorder issues/PRs, but Projects let you also sort into columns, eg for
>>>>>>>> priority. The project UI is very different. But the most important feature
>>>>>>>> is that you can add note cards, not just issues. They can also contain
>>>>>>>> issues from multiple repositories, although project-issue crosslinking only
>>>>>>>> works if project and issue are in the same repo or it's a project in the
>>>>>>>> 'ohrrpgce' organisation and the issue is in a repo under the ohrrpgce
>>>>>>>> organisation**.
>>>>>>>>
>>>>>>>> Compare:
>>>>>>>> https://github.com/ohrrpgce/ohrrpgce/milestone/1
>>>>>>>> https://github.com/orgs/ohrrpgce/projects/1
>>>>>>>>
>>>>>>>> But milestones and projects seem pretty redundant to each other. I
>>>>>>>> wonder whether to use milestones at all. (Maybe just to tag closed bugs,
>>>>>>>> not as a todo list for a release?)
>>>>>>>>
>>>>>>>> ** Adding an issue to a project crosslinks it from the issue
>>>>>>>> itself, see eg https://github.com/ohrrpgce/ohrrpgce/issues/5.
>>>>>>>> We might want to add more repos to the ohrrpgce organisation, so
>>>>>>>> could make sense to put the projects there instead of the repo. For
>>>>>>>> example, I'm thinking of splitting 'web' and 'tools' branches out to
>>>>>>>> separate git repos, since it's pretty inconvenient and strange to have them
>>>>>>>> as orphaned branches.
>>>>>>>> I'm also wondering whether a separate repo where crash reports are
>>>>>>>> automatically uploaded to could be a handy way to manage them.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, 8 Feb 2020 at 01:51, James Paige <Bob at hamsterrepublic.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> That is even better!
>>>>>>>>>
>>>>>>>>> On Fri, Feb 7, 2020, 4:50 AM Ralph Versteegen <teeemcee at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> https://rpg.hamsterrepublic.com/bugzilla/ already exists (it's a
>>>>>>>>>> redirect to the sf buglist, plus hosts
>>>>>>>>>> https://rpg.hamsterrepublic.com/bugzilla/buglist.txt). So I
>>>>>>>>>> guess the logical URL to use is
>>>>>>>>>> https://rpg.hamsterrepublic.com/bugzilla/old-attachments/
>>>>>>>>>>
>>>>>>>>>> <https://rpg.hamsterrepublic.com/old-bugzilla-attachments/>
>>>>>>>>>>
>>>>>>>>>> On Fri, 7 Feb 2020 at 14:40, James Paige <Bob at hamsterrepublic.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Oops! Yes!
>>>>>>>>>>>
>>>>>>>>>>> How about putting them at
>>>>>>>>>>> https://rpg.hamsterrepublic.com/old-bugzilla-attachments/ ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Feb 6, 2020, 5:53 PM Ralph Versteegen <
>>>>>>>>>>> teeemcee at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Oh, right! The main reason I wrote that email was to ask about
>>>>>>>>>>>> uploading the bug attachments somewhere. I see there are still various
>>>>>>>>>>>> redirects and remnants of bugzilla left behind.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, 7 Feb 2020 at 02:47, Ralph Versteegen <
>>>>>>>>>>>> teeemcee at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I agree that too many labels aren't helpful. Labels are most
>>>>>>>>>>>>> useful when you can't just do a text search (all crash bugs will be easily
>>>>>>>>>>>>> found with a search for "crash"), eg for bug priorities.
>>>>>>>>>>>>> I think maybe I'll combine "cosmetic" and "minor" into a
>>>>>>>>>>>>> single "minor" tag, since we likely want that one anyway while "cosmetic"
>>>>>>>>>>>>> is too specific. Wait, how much difference is there between "minor" and
>>>>>>>>>>>>> "needs improvement"?  Plenty of outright bugs are merely cosmetic. Argh!
>>>>>>>>>>>>> Really, "bug" is a region of the continuum (nearly the whole
>>>>>>>>>>>>> continuum) from feature request to "needs improvement" to blocker.
>>>>>>>>>>>>>
>>>>>>>>>>>>> But... major/high priority bugs is possible the most likely
>>>>>>>>>>>>> thing of all that we would want to categorise and search for. And they
>>>>>>>>>>>>> aren't necessarily going to be put on the todo list for the next release
>>>>>>>>>>>>> (milestone). So I guess I should map major/critical/blocker to "major".
>>>>>>>>>>>>> Although the categorisation of bug severities on bugzilla is pretty
>>>>>>>>>>>>> dubious, it is a useful starting point.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 7d344553-34f0-0310-a9b1-970ce8f1c3a2 is the "Repository UUID"
>>>>>>>>>>>>> you see when running "svn info".
>>>>>>>>>>>>> It's also possible to give git-svn a file with mapping from
>>>>>>>>>>>>> username to actual email, but that won't apply retroactively.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, 7 Feb 2020 at 02:08, James Paige <
>>>>>>>>>>>>> Bob at hamsterrepublic.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cool!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I like the idea of rel: and resolved: prefixes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I vote for eliminating the "bug" tag completely. In general I
>>>>>>>>>>>>>> say err on the side of fewer tags when the specific need for a tag is not
>>>>>>>>>>>>>> obvious.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That GitHub contributions trick is cool. I'll try it. Is
>>>>>>>>>>>>>> 7d344553-34f0-0310-a9b1-970ce8f1c3a2 the internal ID of our
>>>>>>>>>>>>>> svn repo?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Feb 6, 2020, 8:04 AM Ralph Versteegen <
>>>>>>>>>>>>>> teeemcee at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, 7 Feb 2020 at 01:43, Ralph Versteegen <
>>>>>>>>>>>>>>> teeemcee at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I've been putting this off for years (two years since my
>>>>>>>>>>>>>>>> last attempt in fact) but I've finally managed to get bugzilla running
>>>>>>>>>>>>>>>> locally and import the databases so that I can export the buglist as xml
>>>>>>>>>>>>>>>> and use this script
>>>>>>>>>>>>>>>> <https://gist.github.com/Zimmi48/d923e52f64fe17c72852d9c148bfcdc6#file-bugzilla2github>
>>>>>>>>>>>>>>>> to import it into github.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm now in the process of customising the script and the
>>>>>>>>>>>>>>>> mapping to labels.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Github doesn't have an API that allows attaching files to
>>>>>>>>>>>>>>>> issue comments, so I suggest that we upload all the attachments to a
>>>>>>>>>>>>>>>> directory on hamsterrepublic.com so that we can link to
>>>>>>>>>>>>>>>> them.
>>>>>>>>>>>>>>>> (Unfortunately I first have to write a script to extract
>>>>>>>>>>>>>>>> them out of the bugzilla DB.)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> With all the labels for components, versions, severity,
>>>>>>>>>>>>>>>> status, etc, we'll have a lot of labels. I think it makes sense to organise
>>>>>>>>>>>>>>>> our labels a bit better. All version labels could be prefixed with "rel: ",
>>>>>>>>>>>>>>>> like "rel: alectormancy". And maybe duplicate, wontfix, invalid,
>>>>>>>>>>>>>>>> worksforme, tooconfusing
>>>>>>>>>>>>>>>> could be prefixed with "resolved:" or "closed:"?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm going to translate the "feature request", "cosmetic",
>>>>>>>>>>>>>>>> "blocker" severities into labels. I think I'll leave behind "major",
>>>>>>>>>>>>>>>> "minor", "normal" and "critical", and the priority levels. They (if not
>>>>>>>>>>>>>>>> default value) will still be in the bug description.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There isn't really anything that corresponds closely to our
>>>>>>>>>>>>>>>> gh "bug" and "needs improvement" labels. I wonder whether we should just
>>>>>>>>>>>>>>>> remove the "bug" label, since although it's informative and useful in
>>>>>>>>>>>>>>>> searches, it's already a nuisance to accurately tag all the bugs** and
>>>>>>>>>>>>>>>> we'll get an extra 1000 bugs missing it. "bug" is basically the absense of
>>>>>>>>>>>>>>>> numerous special categories like "new feature", "invalid" and "tracking".
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Or, I could just apply "bug" as a guess according to
>>>>>>>>>>>>>>> something like that simple rule, and we can fix up any mistakes. Will be
>>>>>>>>>>>>>>> looking through all the open imported bugs anyway.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> **See
>>>>>>>>>>>>>>>> https://github.com/ohrrpgce/ohrrpgce/issues?utf8=%E2%9C%93&q=-label%3Abug+-label%3A%22needs+improvement%22++-label%3A%22new+feature%22
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Bugzilla hides people's email addresses (by stripping the
>>>>>>>>>>>>>>>> domain) by default so user names will be something like "David Gowers
>>>>>>>>>>>>>>>> <00ai99>". I think that's fine. I can (am planning to) also replace the
>>>>>>>>>>>>>>>> username with a github @ mention (it shouldn't cause any notifications,
>>>>>>>>>>>>>>>> because this is the mass import API), but the only people who I know the
>>>>>>>>>>>>>>>> github username of are myself, James and Mike. Doesn't matter.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> BTW, I found that it's possible to make svn commits copied
>>>>>>>>>>>>>>>> to the git-svn mirror on github count towards your "contributions" shown on
>>>>>>>>>>>>>>>> your github profile page. To do so, add a new email address to your github
>>>>>>>>>>>>>>>> account with the form svncommitername at 7d344553-34f0-0310-a9b1-970ce8f1c3a2,
>>>>>>>>>>>>>>>> where svncommitername is e.g. teeemcee or james. The email can be private.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>>>
>>>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>>
>>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>>
>>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Ohrrpgce mailing list
>>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>>
>>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Ohrrpgce mailing list
>>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>>
>>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ohrrpgce mailing list
>>>>>>> ohrrpgce at lists.motherhamster.org
>>>>>>>
>>>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>>>
>>>>>> _______________________________________________
>>>>> Ohrrpgce mailing list
>>>>> ohrrpgce at lists.motherhamster.org
>>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>>
>>>> _______________________________________________
>>>> Ohrrpgce mailing list
>>>> ohrrpgce at lists.motherhamster.org
>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>
>>> _______________________________________________
>> Ohrrpgce mailing list
>> ohrrpgce at lists.motherhamster.org
>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20200314/61c97cb7/attachment-0001.html>

More information about the Ohrrpgce mailing list