Linux nerd and consultant. Sci-fi, comedy, and podcast author. Former Katsucon president, former roller derby bouncer. http://punkwalrus.net

  • 0 Posts
  • 67 Comments
Joined 1 year ago
cake
Cake day: June 22nd, 2023

help-circle

  • I have had two tech jobs like that, even before COVID, starting in 2016. The first time, it was a company that outgrew their workspace. They put us in ‘rent-an-office’ spaces for a bit, and then my boss started working from home a few days a week. Then he allowed me to. We moved to a new office, but it was always empty in my section. That was fine, too, but the commute was terrible, so I started doing 2 days a week, then once a week, then a few times a month. I rarely saw my other coworkers in person, and nobody said anything aloud.

    The next job started because of COVID, and when they started doing RTO, they also wanted to do “hot desking” (no assigned seating) and open office plans, and I was not having that. I was not going to work in a “cafeteria” like setting. So I got contracted work and have worked from home 100% for several years now. Nobody has office space, and we work all over the world to collaborate. I get paid very well.

    I hope i never had to go back to an office. I reach retirement age in about 15 years, and I am hoping to make it.


  • Probably HR (or the NCS equivalent) never told the right people. I am not saying this is actually what happened, but a lot of IT bemoan the fact they are never told some rando employee was fired because HR neglects to inform them. Sometimes it takes months to discover, and even with a 90 day password/login lockout, some halfway decent admin could get around this by secretly building a back door, and using the messed up communication and politics between departments to hide this. Even in the 1990s, I saw people put in “time bombs” in their code that “if such and such is not updated in 6 months, run destructo-script A.”

    But imagine someone like Kandula Nagaraju here. Worked in QA, probably did a great jobs with some skills, but had the personality of swallowing broken glass. He was terminated in October 2022 due to “poor work performance,” which could mean anything. “Not a team player.” Or maybe he really was an idiot: I mean, a smart person would have a conniption, but get employed elsewhere and then slam his former company at parties. “Those NCS folks didn’t know what they had with me!” But this guy was probably someone with some anger management issues, probably a jerk, and possibly stupid. He might have had revenge fantasies, and set up a small virtual server posing as a backup code mirror. But outside the audits, it allowed ssh from the outside, and hid it through a knockd daemon. Or maybe only launched ssh at certain hours before shutting it down again. Silently working away in a sea of virtual servers with little to no updated documentation. He gets in, has internal access, and runs a script with admin credentials because they don’t rotate their AWS keys/secrets quickly enough. Or didn’t even know he was let go.

    After Kandula’s contract was terminated and he arrived back in India, he used his laptop to gain unauthorised access to the system using the administrator login credentials. He did so on six occasions between Jan 6 and Jan 17, 2023.

    That’s embarrassing to the company. Not only did he get in, but SIX TIMES after he was let go. he probably knew what order to run the delete commands (like, say, an aws “terminate-instances” cli command from a primary node), and did so one by one, probably during hours with the least amount of supervision, where the first few alerts would take hours to get someone in the monitoring chain to wake an admin. Given his last day was in November, and he got back in January, the admins probably thought their 90 access credential rotation was “good enough,” but he got in on his 80th day or whatever.

    I know this because I have had to do triage when a former contractor did this to a company I worked for. But instead of wiping out instances, he opened a new set of cloud accounts from the master account, put them in an unmonitored region (in this case, Asia), and spun up thousands of instances to run bitcoin mining. Only because AWS notified us of “unusual traffic” were we made aware at all, and this guy knew his shit and covered his tracks very well. He did it at a speed that could have only been automated. Thankfully, AWS did not charge us the seven figure amount that this activity amassed in just three days.


  • I remember hearing that some Hollywood contracts require that if you sign up for some studio, you must make X amount of films. Big stars get to chose those films to some degree, but once in a while, they have to do “a stinker” to end the contract as “X amount of films done, okay?” or something. Contractual Obligation and all. This film feels like a dumping ground of a lot of those contractual obligation hires from the trailer alone.


    1. Things like CNC machines and proprietary interfaces to TOL equipment, like bus fare systems, message boards, etc.
    2. Don’t connect them to the Internet (most can’t, anyway, but some systems use a run-of-the-mill PC, so…)
    3. Don’t install anything on them that wasn’t supposed to be installed, even wallpaper as this could fuck up the resolution of a small 240 x 180 screen


  • The thing is that for a majority of cases, this is all one needs to know about git for their job. Knowing git add, git -m commit “Change text”, git push, git branch, git checkout , is most of what a lone programmer does on their code.

    Where it gets complicated real fast is collaboration on the same branch. Merge conflicts, outdated pulls, “clever shortcuts,” hacks done by programmers who “kindof” know git at an advanced level, those who don’t understand “least surprise,” and those who cut and paste fixes from Stackexchange or ChatGPT. Plus who has admin access to “undo your changes” so all that work you did and pushed is erased and there’s no record of it anymore. And egos of programmers who refuse any changes you make for weird esoteric reasons. I had a programmer lead who rejected any and all code with comments “because I like clean code. If it’s not in the git log, it’s not a comment.” And his git comments were frustratingly vague and brief. “Fixed issue with ssl python libs,” or “Minor bugfixes.”


  • I was burned afoul by a former admin who, instead of diagnosing why a mail service was failing, labeled a script as a /etc/cron.d file entry as “…” (three dots) which, unless you were careful, you’d never notice in an "ls " listing casually. The cron job ran a script with a similar name which he ran once every 5 minutes. It would launch the mail service, but simultaneous services were not allowed to run on the same box, so if it was running, nothing would happen, although this later explained hundreds of “[program] service is already running” errors in our logs. It was every 5 minutes because our solarwinds check would only notice if the service had been down for 5 minutes. The reason why the service was crashing was later fixed in a patch, but nobody knew about this little “helper” script for years.

    Until one day, we had a service failover from primary to backup. Normally, we had two mail servers servers behind a load balancer. It would serve only the IP that was reporting as up. Before, we manually disabled the other network port, but this time, that step was forgotten, so BOTH IPs were listening. We shut down the primary mail service, but after 5 minutes, it came back up. The mail software would sync all the mail from one server to the other (like primary to backup, or reversed, but one way only). With both up, the load balancer just sent traffic to a random one.

    So now, both IPs received and sent mail, along with web interface users could use. But now, with mail going to both, it created mass confusion, and the mailbox sync was copying from backup to primary. Mail would appear and disappear randomly, and if it disappeared, it was because backup was syncing to primary. It was slow, and the first people to notice were the scant IMAP customers over the next several days. Those customers were always complaining because they had old and cranky systems, and our weekend customer service just told them to wait until Monday. But then more and more POP3 customers started to notice, and after 5 days had passed, we figured out what had happened. And we only did Netbackups every week, so now thousands of legitimate emails were lost for good over 3000 customers. A lot of them were lawyers.

    Oof.


  • Not just LinkedIn profiles: there was a case out here near DC a while ago where a well known company leased out their function space for training meetings. Using a compromised company account, a set of scammers set up some fake recruitment profiles, leased out the meeting space for “software training,” and did some “mass hiring” where 30 individuals had their credentials scanned and duplicated. The effect was someone from the recruiting company was contacting you, you had a face-to-face where you got offered an in-person, you showed up to their offices, and got a “job offer pending a background check,” with a date of hire in official-looking emails. You sent in your SSN, copies of your passport and driver’s licence, and after a few weeks, they tell you to show up for orientation. Only, the day these people showed up, the company was confused and had never heard of you. The people you supposedly spoke to had never heard of you. And your identity was stolen, and huge loans and charges started showing up in your credit report.

    Yikes.


  • Having moderated forums back in the day, I can answer to some of that motivation.

    First, some people are just bullies. A sense of tribalism forms around bullies, who feel the need to act out and repeat the abuses they have endured. Hazing stems from this, too. Cruelty masked as “you should know better,” advice. Given too late.

    Some have a smug sense of superiority, and want to keep it that way. Less smart people means they stay king of the mountain. Others are scared their own lack of knowledge will cripple them if they don’t keep the potential competition down. Insecurities drown out any sense of empathy.

    Some people hate themselves so they punish others in retaliation. Like, trying to erase past cringe by making others hurt to even the score.

    A few are sick of “the same fucking newbie questions again and again and again,” but still hang out in newbie forums for some reason.


  • really just doesn’t do what I needed to do.

    This has been my experience, or sort of does what I want it to do, but I have to rethink what I need it to do instead of something really simple. Like a “new type of shared file system” that replaces NFS/Windows sharing. So instead of files in a standard file system one can manage with a file browser, it has “indexed” your files in such a way that the actual files are renamed into data chunks, and one “finds” files by their non-intuitive search engine that can’t do even basic search engine tricks like “AND/OR” searches, wildcards, and the results are hit and miss. “But it’s faster and more elegant!” So how do you restore from backup when the system fails? “When the system does whatnow?”

    Yeah, no thanks. I can recover files from a file system much easier than some proprietary encoded bullshit fronted with a bad search engine over a proprietary and buggy index.



  • I hate to be honest, but I used Amazon Prime a lot because:

    1. I cannot drive. Thus, getting to the store is difficult.
    2. I must bring in 3-4 items a week, so yeah, I save on shipping.
    3. Auto-subscriptions save a little.
    4. I have priced a lot of stuff over the years, and while Amazon is not always the best, the convenience is impressive.
    5. They have, multiple times, been incredibly helpful with customer service. Like above and beyond.
    6. COVID and nobody masks around here. I have an autoimmune condition, so it’s important that I not leave unless it’s a medical appointment or similar need.
    7. They just have stuff I can’t find anywhere. Yes, as some have said, caveat emptor, but that’s true for all the stores.

    I also save a shit ton of money. When I used to browse Walmart or Target, I used to buy a lot of shit I didn’t need. I don’t get as distracted with focused buying. I also order from Aliexpress if I can wait 30 days, and I have only been ripped off three times in several years, for a total of maybe $35.

    I’m not saying my way is better, and certainly not if it’s better for you, but it’s been a godsend to the house-bound.



  • I have had three Aliexpress orders go bad, and two of them I was refunded. One of them I was not, despite the fact their own shipping still showed it as on it’s way to New Jersey. They kept insisting I “pick it up at the local post office.” I pointed out Virginia was not near New Jersey. I was denied, but only out $8. As far as Amazon vs. Aliexpress, I will agree with you about Amazon customer service being better. But I have had “AliExpress shipping times” on some Amazon products, which led me to believe that the particular store was outside their local jurisdiction.


  • This has been my experience as well with Aliexpress. I know there’s a huge caveat emptor going on, but it’s like you said about batteries and storage, be careful unless you know exactly what you’re buying. Like a 64TB SSD for $15.99 proooobably not the real thing. But I have gotten a lot of SBCs, some stuffed animals, and cheap costume jewelry for my wife (who knows it’s cheap, but doesn’t care). Weirdest “quality” purchase? My “gold colored” tungsten wedding ring was $10, comfort fit, and in 6 years still looks like the real thing. But didn’t dent like my original wedding ring (which is why I needed a replacement, got smashed in a door accident). I’d never buy anything that I knew might cause a fire (like batteries) or possibly poison me (like pills).

    Wish and Temu ripped me off in some of my first purchases. I was only out $30, and I know it was possible, so I just deleted my account and the apps.