●● IRC: #boycottnovell @ Techrights IRC Network: Monday, April 11, 2022 ●● ● Apr 11 [00:49] schestowitz-TR apache not responding [00:49] schestowitz-TR restarting... ● Apr 11 [01:54] schestowitz-TR added tc also to video (latest) page [01:54] schestowitz-TR made complete backup of tuxmachines /var/www to the new laptop ● Apr 11 [02:47] *u-amarsh04 has quit (Quit: Konversation terminated!) [02:47] *u-amarsh04 has quit (Quit: Konversation terminated!) [02:54] *u-amarsh04 (~amarsh04@v6xmmrhxmbafc.irc) has joined #boycottnovell [02:54] *u-amarsh04 (~amarsh04@joseon-rmogvn.g0d7.dtdf.mc4289.IP) has joined #boycottnovell ● Apr 11 [05:06] *DaemonFC has quit (Quit: Leaving) ● Apr 11 [06:47] *Despatche has quit (Quit: Read error: Connection reset by deer) ● Apr 11 [09:00] *schestowitz-TR has quit (Ping timeout: 120 seconds) [09:01] *schestowitz-TR2 (~acer-box@freenode/user/schestowitz) has joined #boycottnovell ● Apr 11 [10:08] schestowitz-TR I lost everything you said after gm [10:10] schestowitz-TR Did you say anything after gm? [10:10] schestowitz-TR I did not read that, so it was cleared before I could see it [10:10] Techrights-sec / home/boycottn/bin/get_dbs.sh ? [10:10] Techrights-sec /home/boycottn/bin/fetch-tm-dump.sh ? [10:10] Techrights-sec Just that there are some scripts which use myqldump [10:10] Techrights-sec just about mysqldump and then the above paths [10:13] schestowitz-TR ah, ok [10:13] schestowitz-TR hand on... [10:13] schestowitz-TR *hang on [10:22] schestowitz-TR it looks like we last ran it last April [10:22] schestowitz-TR I will give it a go now [10:22] schestowitz-TR just checked the script one by line [10:22] schestowitz-TR it ran OK, it moved too fast for me to catch any repairs [10:22] schestowitz-TR but looking at the log it looks like about once a month [10:22] schestowitz-TR there is a bit of a struggle with sort ####################################### [10:22] schestowitz-TR maybe that can explain why the DB deflates a bit Ytalk Error # [10:22] schestowitz-TR and, in this case, does not register every visit [10:22] schestowitz-TR either way, the core bits are intact [10:22] schestowitz-TR I spent a lot of time yesterday [10:22] schestowitz-TR checking tons of things in case this was a symptom of a real issue [10:22] schestowitz-TR I will keep an eye now on hit counters to see if they're back to how they used to be. [10:22] Techrights-sec ack [10:24] schestowitz-TR ftr, as I was saying earlier, it all started when I saw unusually slow number of views [10:24] schestowitz-TR apachelog made no sense, or rather apache top [10:24] schestowitz-TR as everything looked normal [10:24] schestowitz-TR I suspected it was likely some locked up table [10:24] schestowitz-TR but waited to see the db dump at 5:13am for its side, so went to sleep [10:24] schestowitz-TR checked to ensure manstodon and stuff all worked OK [10:24] schestowitz-TR it would be a shame to lose so much data [10:46] Techrights-sec what is the URL for the counts in question? [10:46] schestowitz-TR the counts are still shown incorrectly [10:46] schestowitz-TR I will try flushing out the cache in Drupal [10:46] schestowitz-TR all pages newer than about 16 hours are affected [10:50] schestowitz-TR tbh, lAST NIGHT i WENT (MENTALLY) through all sorts of explanation [10:50] schestowitz-TR also technically checked some [10:50] schestowitz-TR I could not prove that anyone was unable to access [10:50] schestowitz-TR so in a sense 'fixing' this is sort of moot [10:50] schestowitz-TR but the worry was, maybe a sympton [10:50] schestowitz-TR maybe we can just forget about it, knowing the DB is in OK shape [10:50] schestowitz-TR there are all sorts of benign explanations I can come up with [10:50] Techrights-sec ack [10:50] Techrights-sec did clearing the cache help? [10:50] schestowitz-TR I dont' regret double checking [10:50] schestowitz-TR I wopuld have regretted not doing so if later, down the line, [10:50] schestowitz-TR something happened [10:50] schestowitz-TR there are almost 170k nodes for 18 years [10:51] schestowitz-TR I have not tried yet [10:52] schestowitz-TR but it seems to be catching up now [10:52] schestowitz-TR there is a time diff between request and drupal db table updating [10:52] schestowitz-TR for example tuxmachines.org/node/163592 [10:52] schestowitz-TR with www. [10:56] Techrights-sec thanks [10:59] schestowitz-TR ok, in tmax now [10:59] schestowitz-TR see what happened around 11am yesterday [10:59] Techrights-sec ok [10:59] Techrights-sec [ERROR] /usr/libexec/mysqld: Sort aborted ● Apr 11 [11:00] schestowitz-TR we got those before [11:00] schestowitz-TR but then innodb had a shutdown [11:02] Techrights-sec also a "Warning" message too [11:02] Techrights-sec happens a lot there [11:05] schestowitz-TR maybe that can happen about once a month but without me noticing [11:05] schestowitz-TR and maybe I make a big deal out of something that [11:05] schestowitz-TR is a longstanding weakness [11:05] schestowitz-TR 11am is not even close to the time I did rsync iirc [11:05] schestowitz-TR but rsync does not go near the DB anyway [11:05] schestowitz-TR so I thought this could only be a coincidence [11:05] schestowitz-TR either way, running db checks once a month would have [11:05] schestowitz-TR been good practice, which you did suggest in the past [11:05] schestowitz-TR and general housekeeping for other stuff [11:05] schestowitz-TR because we rarely study what really happens silently at [11:05] schestowitz-TR the back end [11:05] Techrights-sec rsync only affects the filesystem load [11:05] Techrights-sec and maybe a tiny amount of CPU [11:16] *psydroid2 (~psydroid@memzbmehf99re.irc) has joined #boycottnovell [11:24] Techrights-sec $ ssh boycottn@tuxmachines.org [11:24] Techrights-sec Unable to negotiate with 23.161.112.115 port 22: no matching host key type found. Their offer: ssh-rsa,ssh-dss [11:24] Techrights-sec that's a less good error involving fully deprecated, insecure algorithms without [11:24] Techrights-sec viable alternatives. An upgrade is long overdue since that change was planned [11:24] Techrights-sec and announced a very long time ago. Can sshd be pulled in from backports? [11:24] schestowitz-TR one option is turn off ssh and go through the hypervisor, which is alpine [11:24] schestowitz-TR but that would interfere with DB backups [11:27] Techrights-sec That's more of a work-around than a real solution. [11:27] Techrights-sec The backups need SSH [11:44] schestowitz-TR easter coming up [11:44] schestowitz-TR I will try to find a good moment to ask kaniini (working [11:44] schestowitz-TR full time) for a new OS on which to build TM [11:44] schestowitz-TR re TM, [11:44] schestowitz-TR I lost a ton of time but have no regrets [11:44] schestowitz-TR because so much time was poured into that database [11:44] schestowitz-TR and I'd rather spend a day posting nothing [11:44] schestowitz-TR just to be sure data is presevred OK [11:44] schestowitz-TR from what I can gather, this is just optics (looks [11:44] schestowitz-TR like low view count) but nothing worth messing about with [11:44] schestowitz-TR and it seems related to innodb, as that started around then [11:44] schestowitz-TR not the first time it happens based on mysql logs [11:44] Techrights-sec Yes, prevention is important. At the risk of nagging, what is the timeline [11:44] Techrights-sec for the OS upgrade? [11:46] schestowitz-TR this week or weekend I will ask kaniini [11:46] schestowitz-TR spinning off a new OS should not be too hard, then we can copy across 'www' [11:46] schestowitz-TR and adding missing packages etc. [11:46] schestowitz-TR the thing that might be a pain is expectation of old DB version [11:46] schestowitz-TR but it's a test machine, so no downtime [11:46] Techrights-sec Thanks! [11:46] Techrights-sec Thursday - Monday are holidays in various places. [11:47] Techrights-sec yes, I was just discussing this with rianne [11:47] Techrights-sec btw, tuxmachines implies tux/linux, so no bsd for OS [11:47] schestowitz-TR yes, I was just discussing this with rianne [11:47] schestowitz-TR btw, tuxmachines implies tux/linux, so no bsd for OS [11:47] Techrights-sec :P [11:48] Techrights-sec not even GNU/kFreeBSD ? [11:49] schestowitz-TR from what I remember it is no longer maintained [11:49] schestowitz-TR figosdev says that devuan quality declined [11:49] schestowitz-TR debian has a hurd version now [11:50] Techrights-sec Too bad, it was an interesting experiment which probably helped the GNU pacakges [11:50] Techrights-sec become more robust. OpenBSD benefits from supporting different hardware [11:50] Techrights-sec architectures, so I figure the effects would be somewhat similar. [11:50] Techrights-sec Hurd is getting closer but I don't think anyone is waiting for it or expecting [11:50] Techrights-sec it any time soon. [11:53] schestowitz-TR btw, re Daily Links [11:53] schestowitz-TR at the moment you have commened out the "line" for IRC posting [11:54] schestowitz-TR Last week I made some changes to speed up posting to IRC [11:54] schestowitz-TR but [11:54] schestowitz-TR I guess that's because you do not run interleave before uploading [11:54] schestowitz-TR as that would add the tags [11:54] schestowitz-TR and I realised a simple solution is, [11:54] schestowitz-TR you comment out the tag part at your end [11:54] schestowitz-TR and I comment out the "line" part [11:54] schestowitz-TR oh,ok! I can sort it out fully at my end [11:54] schestowitz-TR by passing it throught interleave when I retreieve it [11:54] schestowitz-TR then again before posting/finalising [11:54] schestowitz-TR leave iut at my end, I'll work on that now [11:54] Techrights-sec yes [11:54] Techrights-sec right, it would have to be run on your end or else be made much more complex [11:54] Techrights-sec It'd probably help to be able to move most of the processing up to the TR server [11:54] Techrights-sec yes [11:54] Techrights-sec though two runs of interleave should be unnecessary ● Apr 11 [12:00] schestowitz-TR there was a severe storm here last week [12:00] schestowitz-TR we had lots of work to do outside [12:00] schestowitz-TR I am posting your links in TMK now [12:00] schestowitz-TR will then work on the Daily Links flow and test that [12:00] schestowitz-TR I will have to do TONS of patent catchup over Easter [12:00] schestowitz-TR hopefully migration for TM also [12:01] Techrights-sec I hope for migration. If there is any way to help or prepare, let me know. [12:05] schestowitz-TR thanks, you can now see your links at top of TM [12:05] schestowitz-TR the stats are vanity anyway [12:05] schestowitz-TR I cannot see evidence of any real harm to data [12:05] schestowitz-TR so I shall leave it alone [12:05] schestowitz-TR there is a mismatch between apacghe and drupal counts [12:05] schestowitz-TR also, of course none of this counts views via front page, RSS feed etc. [12:05] schestowitz-TR so it's a case of assuring myself the DB was in a good state [12:05] schestowitz-TR first with the backup [12:05] schestowitz-TR then scanning it [12:05] schestowitz-TR OS migration can ameliorate many things [12:05] schestowitz-TR but it would also be a pain in transitiob [12:05] Techrights-sec ack [12:05] Techrights-sec yes [12:57] *psydroid2 has quit (connection closed) ● Apr 11 [13:00] *psydroid2 (~psydroid@memzbmehf99re.irc) has joined #boycottnovell [13:29] schestowitz-TR i am learning some perl along the way [13:29] schestowitz-TR but I got stuck [13:29] schestowitz-TR I am trying to push the "irc line" before