Its probably AI bots that are crawling the website.. I've had this happen to one of my websites that was slow and traffic spiked... was Chatgpt, Meta, AWS, Perplexity bots etc
its been a minute since i ran an httpd server (im a nginx fan over apache) but does robots.txt not work anymore?
it was never perfect but between robots.txt and a proper .htaccess file my sites never got banged too much. but man i had to cover EVERY directory. i had uploaded a hard to find music video and somebody found it cos i checked my server logs and the file had been downloaded thousands of times. figure the file itself was about 180MB so it was a costly mistake lol
with these new ai bots im glad to step back. people ruin everything. i have no idea why anybody WANTS ai to exist
years ago i was obsessed with Cobe/megaHAL and its scary to see how much artificial "intelligence" has progressed in just 10 years
A Markov chain based text generation library and MegaHAL style chatbot - pteichman/cobe
github.com