So I have a bucket list…
Lately I’ve been slowly but surely ticking things off my bucket list…you know, the list of things I want to get done before I pass on the the next existence, whatever that happens to be. Thought maybe I’d share some of my progress to date.
- Played in a World Series of Poker event – check, although I may try again next year
- Getting my college degree – in progress
- Publish a book – check, although a second one is in the works
- Cruise to Alaska -someday
- Another vacation on Maui – someday
I’ll keep adding to my bucket list as I go but I’ve got things moving along!
One of the things I am struggling with is whether or not I’ll walk when (if?) I graduate towards the end of 2013 as I expect to. It would require a trip to Salt Lake City to do so, so I am not sure how many of my family could make it, but it might be worthwhile to do. What do you think?
Why can’t web crawlers use some ethics…and some intelligence?
So I’ve been up since 4am this morning battling what is essentially a Distributed Denial of Service attack…basically a bunch of computers sending requests to our web servers over and over and over again. After two hours of battling, the culprit was found and disabled.
This company offers to crawl data on websites via some customizable code. However, their business practices are definitely questionable. A google search is most enlightening. This web crawler hit our site over 7,000 times in a 10 minute span. And based on that Google search, we are not the only ones.
Now there are a couple of things I simply don’t understand. First of all, who’s the genius at 80legs.com that thinks hitting any site on the web at this volume is a good idea? I understand that they have a business, that they are selling crawling technology, but how much do they expect to sell if the end result of implementing their crawler is the immediate block of the crawler by the unwitting victim? Certainly whoever is paying them to crawl our site is now going to be disappointed.
Second, why would anyone think that this sort of crawling is ethical in this day and age of botnets and hackers? If I was building a business based on this technology I would at a minimum make sure targets could remove themselves from the line of fire (80legs claims it does so but it doesn’t work…they don’t honor robots.txt like they say the do), and make sure my bot speed was within reason speed wise. Google, Bing, and Yahoo all crawl the web without causing mass chaos and overwhelmed servers. Certainly if you have the intelligence to write a crawler, you have the intelligence to throttle a crawler.
Or maybe my standards are too high.
Last Night’s Crazy Dream
So last night I had one of those crazy dreams…I was working at some major retailer in some major city, professing my undying affection for one my high school crushes, who had just arrived in the store in a panic-room style elevator. She was returning my advances, only to tell me she was part of a revolution about to attack the city, from the basement, with an army of high tech weapons and a bunch of revolutionary friends, and she asked me to join her, which, of course, I did “for love” or some such sentimental reason.
Then Elijah woke up thirsty and wanted his water refilled. At 4:30am. Sweet.
One of those real “real” dreams where you think it’s really happening, and the only reason I’m posting it is because it was so crazy …and it sounds like the plot to a great story or maybe even book, and I don’t want to forget it…
P.S. I was a lot younger in the dream too…and even had all my hair back. Been having a lot of those lately.
Be welcome and true.
Rob Hope 