[Ohrrpgce] Wiki down

James Paige Bob at hamsterrepublic.com
Sat Jan 28 21:38:04 PST 2017


I am not sure. If the outage had started at 5am Pacific time, that would
have been a smoking gun, since that is when the update from job runs... But
strangely the last time I got cron mail for that job was 2 days ago.

On Saturday, January 28, 2017, Ralph Versteegen <teeemcee at gmail.com> wrote:

> Great. But was the outage originally caused by the update script failing
> and leaving something in a bad state, or did you only try to run it to fix
> an unexplained problem?
>
> On 29 January 2017 at 18:01, James Paige <Bob at hamsterrepublic.com
> <javascript:_e(%7B%7D,'cvml','Bob at hamsterrepublic.com');>> wrote:
>
>> Good news!
>>
>> I was able to fix it.
>>
>> I suspect that dreamhost's annoying memory-usage-watchdogs happened to
>> kill the wiki update script at a very bad moment when it was half-way
>> through updating the schema of the user_groups table.
>>
>> I was able to tweak the update script to fix the problem, and everything
>> seems to be working fine now.
>>
>>
>>
>> On Sat, Jan 28, 2017 at 5:52 PM, James Paige <Bob at hamsterrepublic.com
>> <javascript:_e(%7B%7D,'cvml','Bob at hamsterrepublic.com');>> wrote:
>>
>>> Yep, I just saw it. I tried re-running the update script,  and ended up
>>> with the following backtrace, but I don't think I will be able to sort out
>>> the actual problem today :(
>>>
>>> james_paige at irwin:~/rpg.hamsterrepublic.com/ohrrpgce$ /usr/bin/php
>>> maintenance/update.php
>>> MediaWiki 1.29.0-alpha Updater
>>>
>>> Your composer.lock file is up to date with current dependencies!
>>> Going to run database updates for ohrrpgce_wiki
>>> Depending on the size of your database this may take a while!
>>> Abort with control-c in the next five seconds (skip this countdown with
>>> --quick) ... 0
>>> Turning off Content Handler DB fields for this part of upgrade.
>>> ...have ipb_id field in ipblocks table.
>>> ...have ipb_expiry field in ipblocks table.
>>> ...already have interwiki table
>>> ...indexes seem up to 20031107 standards.
>>> ...have rc_type field in recentchanges table.
>>> ...index new_name_timestamp already set on recentchanges table.
>>> ...have user_real_name field in user table.
>>> ...querycache table already exists.
>>> ...objectcache table already exists.
>>> ...categorylinks table already exists.
>>> ...have pagelinks; skipping old links table updates
>>> ...il_from OK
>>> ...have rc_ip field in recentchanges table.
>>> ...index PRIMARY already set on image table.
>>> ...have rc_id field in recentchanges table.
>>> ...have rc_patrolled field in recentchanges table.
>>> ...logging table already exists.
>>> ...have user_token field in user table.
>>> ...have wl_notificationtimestamp field in watchlist table.
>>> ...watchlist talk page rows already present.
>>> ...user table does not contain user_emailauthenticationtimestamp field.
>>> ...page table already exists.
>>> ...have log_params field in logging table.
>>> ...logging table has correct log_title encoding.
>>> ...have ar_rev_id field in archive table.
>>> ...have page_len field in page table.
>>> ...revision table does not contain inverse_timestamp field.
>>> ...have rev_text_id field in revision table.
>>> ...have rev_deleted field in revision table.
>>> ...have img_width field in image table.
>>> ...have img_metadata field in image table.
>>> ...have user_email_token field in user table.
>>> ...have ar_text_id field in archive table.
>>> ...page_namespace is already a full int (int(11)).
>>> ...ar_namespace is already a full int (int(11)).
>>> ...rc_namespace is already a full int (int(11)).
>>> ...wl_namespace is already a full int (int(11)).
>>> ...qc_namespace is already a full int (int(11)).
>>> ...log_namespace is already a full int (int(11)).
>>> ...have img_media_type field in image table.
>>> ...already have pagelinks table.
>>> ...image table does not contain img_type field.
>>> ...already have unique user_name index.
>>> ...user_groups table exists and is in current format.
>>> ...have ss_total_pages field in site_stats table.
>>> ...user_newtalk table already exists.
>>> ...transcache table already exists.
>>> ...have iw_trans field in interwiki table.
>>> ...wl_notificationtimestamp is already nullable.
>>> ...index times already set on logging table.
>>> ...have ipb_range_start field in ipblocks table.
>>> ...no page_random rows needed to be set
>>> ...have user_registration field in user table.
>>> ...templatelinks table already exists
>>> ...externallinks table already exists.
>>> ...job table already exists.
>>> ...have ss_images field in site_stats table.
>>> ...langlinks table already exists.
>>> ...querycache_info table already exists.
>>> ...filearchive table already exists.
>>> ...have ipb_anon_only field in ipblocks table.
>>> ...index rc_ns_usertext already set on recentchanges table.
>>> ...index rc_user_text already set on recentchanges table.
>>> ...have user_newpass_time field in user table.
>>> ...redirect table already exists.
>>> ...querycachetwo table already exists.
>>> ...have ipb_enable_autoblock field in ipblocks table.
>>> ...index pl_namespace on table pagelinks includes field pl_from.
>>> ...index tl_namespace on table templatelinks includes field tl_from.
>>> ...index il_to on table imagelinks includes field il_from.
>>> ...have rc_old_len field in recentchanges table.
>>> ...have user_editcount field in user table.
>>> ...page_restrictions table already exists.
>>> ...have log_id field in logging table.
>>> ...have rev_parent_id field in revision table.
>>> ...have pr_id field in page_restrictions table.
>>> ...have rev_len field in revision table.
>>> ...have rc_deleted field in recentchanges table.
>>> ...have log_deleted field in logging table.
>>> ...have ar_deleted field in archive table.
>>> ...have ipb_deleted field in ipblocks table.
>>> ...have fa_deleted field in filearchive table.
>>> ...have ar_len field in archive table.
>>> ...have ipb_block_email field in ipblocks table.
>>> ...index cl_sortkey on table categorylinks includes field cl_from.
>>> ...have oi_metadata field in oldimage table.
>>> ...index usertext_timestamp already set on archive table.
>>> ...index img_usertext_timestamp already set on image table.
>>> ...index oi_usertext_timestamp already set on oldimage table.
>>> ...have ar_page_id field in archive table.
>>> ...have img_sha1 field in image table.
>>> ...protected_titles table already exists.
>>> ...have ipb_by_text field in ipblocks table.
>>> ...page_props table already exists.
>>> ...updatelog table already exists.
>>> ...category table already exists.
>>> ...category table already populated.
>>> ...have ar_parent_id field in archive table.
>>> ...have user_last_timestamp field in user_newtalk table.
>>> ...protected_titles table has correct pt_title encoding.
>>> ...have ss_active_users field in site_stats table.
>>> ...ss_active_users user count set...
>>> ...have ipb_allow_usertalk field in ipblocks table.
>>> ...change_tag table already exists.
>>> ...tag_summary table already exists.
>>> ...valid_tag table already exists.
>>> ...user_properties table already exists.
>>> ...log_search table already exists.
>>> ...have log_user_text field in logging table.
>>> ...l10n_cache table already exists.
>>> ...index ls_field_val already set on log_search table.
>>> ...index change_tag_rc_tag already set on change_tag table.
>>> ...have rd_interwiki field in redirect table.
>>> ...transcache tc_time already converted.
>>> ...*_mime_minor fields are already long enough.
>>> ...iwlinks table already exists.
>>> ...index iwl_prefix_title_from already set on iwlinks table.
>>> ...have ul_value field in updatelog table.
>>> ...have iw_api field in interwiki table.
>>> ...iwl_prefix key doesn't exist.
>>> ...have cl_collation field in categorylinks table.
>>> ...categorylinks up-to-date.
>>> ...module_deps table already exists.
>>> ...ar_page_revid key doesn't exist.
>>> ...index ar_revid already set on archive table.
>>> ...ll_lang is up-to-date.
>>> ...user_last_timestamp is already nullable.
>>> ...index user_email already set on user table.
>>> ...up_property in table user_properties already modified by patch
>>> patch-up_property.sql.
>>> ...uploadstash table already exists.
>>> ...user_former_groups table already exists.
>>> ...index type_action already set on logging table.
>>> ...have rev_sha1 field in revision table.
>>> ...batch conversion of user_options: nothing to migrate. done.
>>> ...user table does not contain user_options field.
>>> ...have ar_sha1 field in archive table.
>>> ...index page_redirect_namespace_len already set on page table.
>>> ...have us_chunk_inx field in uploadstash table.
>>> ...have job_timestamp field in job table.
>>> ...index page_user_timestamp already set on revision table.
>>> ...have ipb_parent_block_id field in ipblocks table.
>>> ...index ipb_parent_block_id already set on ipblocks table.
>>> ...category table does not contain cat_hidden field.
>>> ...have rev_content_format field in revision table.
>>> ...have rev_content_model field in revision table.
>>> ...have ar_content_format field in archive table.
>>> ...have ar_content_model field in archive table.
>>> ...have page_content_model field in page table.
>>> Content Handler DB fields should be usable now.
>>> ...site_stats table does not contain ss_admins field.
>>> ...recentchanges table does not contain rc_moved_to_title field.
>>> ...sites table already exists.
>>> ...have fa_sha1 field in filearchive table.
>>> ...have job_token field in job table.
>>> ...have job_attempts field in job table.
>>> ...have us_props field in uploadstash table.
>>> ...ug_group in table user_groups already modified by patch
>>> patch-ug_group-length-increase-255.sql.
>>> ...ufg_group in table user_former_groups already modified by patch
>>> patch-ufg_group-length-increase-255.sql.
>>> ...index pp_propname_page already set on page_props table.
>>> ...index img_media_mime already set on image table.
>>> ...iwl_prefix_title_from index is already non-UNIQUE.
>>> ...index iwl_prefix_from_title already set on iwlinks table.
>>> ...have ar_id field in archive table.
>>> ...have el_id field in externallinks table.
>>> ...have rc_source field in recentchanges table.
>>> ...index log_user_text_type_time already set on logging table.
>>> ...index log_user_text_time already set on logging table.
>>> ...have page_links_updated field in page table.
>>> ...have user_password_expires field in user table.
>>> ...have pp_sortkey field in page_props table.
>>> ...recentchanges table does not contain rc_cur_time field.
>>> ...index wl_user_notificationtimestamp already set on watchlist table.
>>> ...have page_lang field in page table.
>>> ...have pl_from_namespace field in pagelinks table.
>>> ...have tl_from_namespace field in templatelinks table.
>>> ...have il_from_namespace field in imagelinks table.
>>> ...img_major_mime in table image already modified by patch
>>> patch-img_major_mime-chemical.sql.
>>> ...oi_major_mime in table oldimage already modified by patch
>>> patch-oi_major_mime-chemical.sql.
>>> ...fa_major_mime in table filearchive already modified by patch
>>> patch-fa_major_mime-chemical.sql.
>>> ...user_id is already unsigned int.
>>> ...rc_comment in table recentchanges already modified by patch
>>> patch-editsummary-length.sql.
>>> ...hitcounter doesn't exist.
>>> ...site_stats table does not contain ss_total_views field.
>>> ...page table does not contain page_counter field.
>>> ...msg_resource_links doesn't exist.
>>> ...msg_resource doesn't exist.
>>> ...bot_passwords table already exists.
>>> ...have wl_id field in watchlist table.
>>> ...cl_collation key doesn't exist.
>>> ...index cl_collation_ext already set on categorylinks table.
>>> Updating category collations...Fixing collation for 51 rows.
>>> Selecting next 100 rows... processing...0 done.
>>> 0 rows processed
>>> ...done.
>>> ...index rc_name_type_patrolled_timestamp already set on recentchanges
>>> table.
>>> ...rev_page_id index not found on revision.
>>> ...pl_namespace, tl_namespace, il_to indices are already non-UNIQUE.
>>> ...have ct_id field in change_tag table.
>>> ...have ts_id field in tag_summary table.
>>> ...rc_ip in table recentchanges already modified by patch
>>> patch-rc_ip_modify.sql.
>>> ...have el_index_60 field in externallinks table.
>>> Adding ug_expiry field to table user_groups
>>> ...[7e6dcc70478a580cb2309cb0] [no req]   DBQueryError from line 1054 of
>>> /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> libs/rdbms/database/Database.php: A database query error has occurred.
>>> Did you forget to run your application's database schema updater after
>>> upgrading?
>>> Query: ALTER TABLE `user_groups`
>>>  DROP INDEX ug_user_group,
>>>  ADD PRIMARY KEY (ug_user, ug_group),
>>>  ADD COLUMN ug_expiry varbinary(14) NULL default NULL,
>>>  ADD INDEX ug_expiry (ug_expiry)
>>>
>>> Function: Database::sourceFile( /home/james_paige/rpg.hamsterr
>>> epublic.com/ohrrpgce/maintenance/archives/patch-user_groups-
>>> ug_expiry.sql )
>>> Error: 1091 Can't DROP 'ug_user_group'; check that column/key exists (
>>> mysql.motherhamster.org)
>>>
>>> Backtrace:
>>> #0 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> libs/rdbms/database/Database.php(912): Database->reportQueryError(string,
>>> integer, string, string, boolean)
>>> #1 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> libs/rdbms/database/Database.php(3120): Database->query(string, string)
>>> #2 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> libs/rdbms/database/Database.php(3069): Database->sourceStream(unknown
>>> type, NULL, NULL, string, NULL)
>>> #3 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> installer/DatabaseUpdater.php(666): Database->sourceFile(string)
>>> #4 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> installer/DatabaseUpdater.php(734): DatabaseUpdater->applyPatch(string,
>>> boolean, string)
>>> #5 [internal function]: DatabaseUpdater->addField(string, string,
>>> string)
>>> #6 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> installer/DatabaseUpdater.php(465): call_user_func_array(array, array)
>>> #7 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/includes/
>>> installer/DatabaseUpdater.php(429): DatabaseUpdater->runUpdates(array,
>>> boolean)
>>> #8 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/maintenan
>>> ce/update.php(172): DatabaseUpdater->doUpdates(array)
>>> #9 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/maintenan
>>> ce/doMaintenance.php(111): UpdateMediaWiki->execute()
>>> #10 /home/james_paige/rpg.hamsterrepublic.com/ohrrpgce/maintenan
>>> ce/update.php(217): require_once(string)
>>> #11 {main}
>>>
>>>
>>> On Sat, Jan 28, 2017 at 5:49 PM, Ralph Versteegen <teeemcee at gmail.com
>>> <javascript:_e(%7B%7D,'cvml','teeemcee at gmail.com');>> wrote:
>>>
>>>> The OHR wiki is down (for at least an hour now), printing out odd error
>>>> messages that I've never seen before. HR.com is fine.
>>>>
>>>> [WI06MNBxxg8AAA6GCoUAAAAN] 2017-01-29 00:41:21: Fatal exception of type
>>>> "DBQueryError"
>>>>
>>>> _______________________________________________
>>>> Ohrrpgce mailing list
>>>> ohrrpgce at lists.motherhamster.org
>>>> <javascript:_e(%7B%7D,'cvml','ohrrpgce at lists.motherhamster.org');>
>>>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Ohrrpgce mailing list
>> ohrrpgce at lists.motherhamster.org
>> <javascript:_e(%7B%7D,'cvml','ohrrpgce at lists.motherhamster.org');>
>> http://lists.motherhamster.org/listinfo.cgi/ohrrpgce-motherhamster.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.motherhamster.org/pipermail/ohrrpgce-motherhamster.org/attachments/20170128/a93c1cf3/attachment-0001.htm>


More information about the Ohrrpgce mailing list