Help size covers?

We've got a new book-display page in the works that needs to know the size of images before it loads them—in order to get the spacing of the "shelf" right.

Unfortunately, LibraryThing hasn't previously collected Amazon-image sizes. We're doing it now, but we need help filling in past images. We can't request all 4-million-odd images ourselves—it would bog down the server, and send our traffic through the roof.

So, I made a page:
http://www.librarything.com/test_imageloading.php

The page loads images in your browser, sizes them and sends back the information to LibraryThing. Then it waits about 10 seconds and does it again.

That's a very manageable load for anyone not on dial-up—not much more data than browsing around the site normally. You'd do us a favor if you left the window open while for an hour, or all day. You'll help us build up the sizing information we need to make the site prettier. If 10, 100 or 1,000 users help us, we can do this in hours or days, not weeks.

Thanks,
Tim

Helping.

Thanks. I added Amazon links to the images, so you can find out more about books, and to mollify Amazon with a few sales, if they don't like us collecting the data this way...

I'll probably cap members to X thousand images, so people don't feel obligated to do it forever.

Meanwhile, however, I'm taking a shower (yes, TMI).

so we just open the page and it does all the work?

if so .. I am helping too.. lol

I'm a big help. :)

On a serious note note since we're talking about the covers and their resizing, when you go into the Add Books tab and do a search, usually the cover thumbnails are the last to load and this usually jumps the page around a bit. I know I've hit the wrong title a few times due to this jumping. Since all the thumbnails in this case are of the same height, can some allowance be made via CSS to allocate this spacing when the output is formatted?

Does that make sense?

>6 theapparatus:

Yes. Exactly. We can do it once we get the info.

T

Ok -- helping

Watching the estimated completion time is mesmerizing, like when watching a download bar. Every page load, it gets earlier by a few minutes... when I started, it was at sometime Tuesday morning (about 10am GMT), and now it's 10pm Monday GMT.

I wish that clicking on a cover opened the amazon page in a new tab/window, but it's not a big deal. It's probably not even intended behavior to have us watching the page as a form of entertainment. :P

Would go quicker if the *cough* connection would stop dropping.

no problems here... it is just running smoothly and quickly in the background.

Bruegger's doesn't have the best connection.

It is interesting to watch what goes by. Looks like government docs now.

i have classics :)

White covers with red trim? That's what I saw.

I noticed though that the timeout is now set to 5k milliseconds or 5 seconds instead of the announced 10 seconds. I peaked wondering if I could stick this up on one of my servers for the day.

edit: Hmm, now it's 6 seconds. Adjustments?

Wow, that's amazing!

Got it running in its own window ... peeking in every now and then to see the books!

 

Kiddie books now. Some easy reader series.

Pity I have blood work though. Going to get to 35k and have to leave.

edit: I did notice Tim's numbers haven't moved for a bit. :)

reedit: Going past million-books.com spam now.

Aw, a limit of 50k? I could've kept running it at home while I'm at work.

It seems to have stuck. Did Amazon apply some brakes?

v

eta: I refreshed it manually and that got it going again.

I'm not much help with my dial-up internet but I'll try to load up a few pages from time to time, hopefully it helps a little. Every little bit counts right? :)

You really want the 50K limit? It was nothing to have this running on a tab in the background...

>22 stephmo: Agree - no problem doing this is the background at all

22,23> My first thought was that it was to limit bandwidth hits from refreshing the page, but that doesn't make much sense, since the 3.9mil would have to be served up eventually anyway.

I'm wondering now if the 50k mightn't be to limit the number of hits on Amazon's servers from any particular IP (if that makes a difference). It could also be to give more folks a chance to "help".

Seeing everyone else's numbers grow so fast makes me realize how horrible my connection is. :o

You should get a special star for even attempting this on dial-up :)

26: Haha, I just timed myself. 2 minutes and 55 seconds for 2 pages... almost 9 times slower than the projected 10 seconder per page!

Note to self: Get a faster internet connection sometime eventually...

Watching covers go by on my iPad while I play WoW. Not a bad way to spend a sticky, humid Sunday.

I know.. kinda mesmerizing..

Stephen dear... slow internet is the devil.. I wish you had high speed..

awww bummer. I am finished. It doesn't want me to do more than 50,000.
Guess I should find something else to look at.. lol

22 I agree...no problem... and when it stops you can't see the number change...
( note to self, *get life*)

26 I agree... He is nice than me...

Very cool!

When you refresh your page, it always adds 50 books to your total -- even if you've passed 50,000 and no more covers are being downloaded!!

My total's now 50,300.

Oops.

ETA: updated total

>33 SilentInAWay: Now I don;t feel bad for having two tabs open, both of them running these. :)

edit: Confirmed that the count goes up by 50 with a hard refresh even though you;re at the 50k limit and nothing is displaying.

Auto-stopped at 50,049.
No idea how I dropped the odd 1, but I guess it helped add another 49!
Looks like we need some new recruits to hit out goal.

If Tim removes the 50,000 limit, twill be done in no time.

Well, that went quick. Threw open a few pages in my browser and had it done within the last couple of hours. I wish it was ip-savvy; I'd throw a few more through when I'm next at the coffeeshop. :D

Just found the thread, helping...

I had the same thought, but sadly it didn't work :(

(really kevmalone, under this heavy disguise)
We have a family Bookmooch (free) account so friends on LT can pre-mooch, and it keeps my personal account clean. It's somewhat moribund, however it allows me to continue with the download.
So the limit seems to be account, rather than ip dependent
I'll keep going, unless someone calls "foul" on this for abuse.

I think having it open in a few tabs (and on the iPad) also wound up making me break 50k more than I would otherwise. Though I did do one refresh that hit the bug mentioned earlier.

40> Well, you might want to hold off on it until Tim pipes up. It's very likely he put on the 50k limit to keep more than 50k from coming from one IP.

:(

Just saw the estimated time of completion increase. Went from 00:46 to 00:47. (But it's back to 00:46 now.)

>40 kevmaloneBM: Will do

>43 keristars: Now at 00:59

It's baa-aack

It's started refreshing itself for me again, but I'm getting this error at the bottom of the righthand column:

Maxed-out
Warning: join(): Bad arguments. in /var/www/html/test_imageloading.php on line 186

Also, most of the covers right now seem to be along the lines of "The 2007 Import and Export Market for Live Poultry Weighing Up to 185 Grams (6.53 Ounces) in Netherlands" and "The 2007 Report on Tufted Carpet and Rugs Made from Polyester Face Yarn: World Market Segmentation by City ".

It's rather ...interesting, I suppose.

(Also, the error message is gone now.)

Okay, I've allowed you to go to 100,000. I'm a little worried that Amazon will get annoyed, although we actually fetch far more covers than this every day. (Every time you hit your catalog, and many other pages.) So, go buy something from Amazon.

Incidentally, I should be doing this by leveraging the catalog. I avoided it because the two JavaScript libraries in question conflict—or at least, I don't know how to get them to play nice. Also, I'm requesting a specific size. Although Amazon is rounding the size somewhere, if I request all the same size, at least they will be rounding in the same way, as it were.

Yeah, I'm getting all those odd books too...but I'm not complaining because they seem to be loading faster. >:)

>47 keristars:

Fixed that error.

I LOVE the "runs" you get. I got about five screens of books like this one



Covering every region of the world. In case you wanted to research Count Chocula distribution in Tamil-Nadu...

now it's doing picture books!!!

ETA: now it's not

I had a run of Golden Books for a while (which was much better than the 5 pages of romance covers I hit early in the process).

It's kind of neat to see how publisher runs work.

At Houghton Mifflin we'd buy huge swaths of ISBNs and then give them internal meanings. So, in theory, if you knew the ISBN for the grade 5 Social Studies book, you could figure out the grade 6 Math book. In theory, except that things got screwed up from time-to-time, as you'd expect. Two books get the same ISBN. Two different ISBNs printed on outside and inside cover, etc. I imagine other publishers have other systems.

This knowledge is, however, all arcane. Nobody publishes or tracks it.

It's interesting how the covers of books I own jump out of the crowd. It's like facial recognition -- I don't have to think or carefully scan the page or anything. Bang! -- there's a book I own.

Reaping the benefits of millenia of evolution, I suppose -- the same skill that helped our ancestors instantly recognize predators...

(I previously posted this to the wrong thread)

Maxed-out!! I like it.

>54 SilentInAWay:

I think it's more like sex partners. Unless you're scared of your books...

>56 timspalding:

you have no idea...

thanks for upping our quota :) I am enjoying it

Tim... it seems to be reloading the same titles and covers..

but the number of loaded covers under my name goes up each time.

ok

Nothing in posting .. not in the forums, not messages... what the heck??
Same covers loading over and over..

#58: Yes, great fun. I wonder how pictures/second is computed, It seems to change fairly slow, so I guess it is looking at the total number of pictures and the total number of seconds since this started. If so it is probably underestimating the current speed a bit. I think the speed is more like 100 covers a second than the 90 reported.
If so, we'll finish an hour earlier,

Of course we'll both max out long before that :-)

>58 mckait:-60

Nice. Someone using Chinese 2 (either simplified or not, I can't remember) hit a Zeitgeist page that had never been cached for Chinese 2. I didn't think there were any such pages left, but apparently there are. Anyway, it slowed down your server something terrible. Apologies.

> I wonder how pictures/second is computed, It seems to change fairly slow

It's random, plus the time to load the page.

whew! it was kinda twilight zone creepy .. I felt like I was here all alone with LT running amok!

>56 timspalding: I'm leaving the predators and sex partners subject alone. My numbers are probably embarrassing. But I wonder how many LT screen names Tim can recognize if he saw them outside of the site?

I've been getting lots and lots of children's books for a while . . .

Whoa! You have to be fast to click on a cover that looks interesting before it disappears and the next batch pops up.

>64 etrainer:

It's going by ISBN, and they're allocated by publisher.

Going again. I noted we now have a maxed out section along the right hand side.

edit: Maybe Amazon won;t notice until their managers come in on Monday morning?

My goodness, that's a lot of Sepia I just hit...

Just got a slab of self-help books, then switched to travel guides.

Running it in two browsers, one for each of my accounts.

I just noticed this so I'm never going to catch up to the big numbers. I'll give it two tabs-worth of my intrabweb pipe and let it run.

Joiedelivre is running through DK Publishing right now - lots of Eyewitness books!

It's not slowing my reading of LT Talk, which is all I'm doing right now (and drinking my second cup of coffee) so I've given it a third tab. Soon have those last million done!

>71 justjim:. Look at it this way. You've done more than 99% of the other folks. :)

edit: What I just got: The 2000-2005 World Outlook for Reconstituted Wood Products. A little light reading maybe?

reedit: Interesting that that work doesn't come up via LT search. ISBN is 0757655572 if anyone wants to check it. My understanding that all of these were books that were already here.

>74 theapparatus:

It's been touched somehow by LT. It could include searched-for, I think.

Okay, see, this is another HUGE time-waster! I swear Tim, you keep putting out stuff like this I'll end up *living* on this site!

I'm running it in the background, but I have to keep obsessively clicking over to that window because omg what if I miss a reall cool-looking book?? lol

You know you're in trouble when essentially random books attract you :)

.... Isn't that what bookstores are all about? :P

I was getting mostly political books, but now it's cycled on to self-help and disabilities.

heh How many bookstores carry a copy of The 2000-2005 World Outlook for Reconstituted Wood Products? :)

edit: Just hit some Peanuts books but also just hit my 100k limit.

All three of the tabs I have open were doing the same publisher - nothing but egg-yolk yellow with white squares - Kessinger, I think.

80: I just got those too!

Ack! *chokes* "The Joy of Sex Toys". Thank goodness mom wasn't in the room. lol

I had a boring run of "_____ for Dummies" books, which then flipped to testing help books for GMAT and SAT. I am now seeing fun kids books. You just never know what is going to crop up next on your screen.....

I saw that (Joy of) - are we doing the same title more than once? If so, it will take a long time to get everything!

Saw the kids books too, but now on to fiction, after a spate of classical music scores.

Medical books now.

Wow! I was AFK for a while so I gave it 5 tabs and an iPad. Under half a million now!

I'm just wondering if it'll wrap up within 12 hours. What was the original predicted time?

Looks like I won't hit the limit - there aren't enough covers left!

man, this is fun, I wish it didn't have an end! lol

Bah! I left it running all night, but it must have stuck about 30 seconds after I left the room...

And this cover is why I need to know the dimensions beforehand!

Question: This is a REALLY awesome way to find random books that look interesting. Is there anything else like this on the site? I mean, more random then recommendations or such...

91: There's a random book thingy but I don't know the link...

I'm racing to try to read 5,000 before all the covers are gone. Blast this slow connection! :)

Whee! 50k for me and less than 37k left to do.

91, 92> Here's the link to random work:

http://www.librarything.com/random.php?type=work

or here.

Hm?

Covers left: -3,017

We're done!
Big thumbs up to Ape for hanging in there.

Aaand, we're done. Images left has gone into negative numbers! I've shut down all my tabs, is there really more to do?

Whoo, I just barely hit 5,000 before the covers ran out. :D

..... and going deeper into the negative.....

Tim, pull the plug!!

the engines cannot take it

helping

This function will self-destruct in 10 seconds...

I went back to the page to see what all the fuss is about. Are w out of covers and repeating old ones again, or is the "covers left" number incorrect? Odd that it would keep going for no reason... *shrug*

I got to over 70k before I shut mine down. Cool beans!

>90 timspalding: I dunno, Tim, I kinda like the effect...sorta surreal.

Kind of reminds me of that function we had a long time ago, where you could watch the titles of all the new additions to LT scroll by ... Quite hypnotic.

Well, I found another 2 million or so covers. But I'm capping people at 25,000 and slowing it down. Let's get some new blood.

(We have all the covers we need for feature X, so we have some time.)

I'm free!

Capped out. Do I at least get a feather in it?

An ostrich plume to you m'lady!

Okay, I am helping, too.

Tim,

Why did you max us out, I was just letting my PC run while I sat and read.

I want to help!

Wanted to give servers a break. We're back if you want to try now.

T

Seems to be loading 25 covers at a time now . . .

It is. Can people run it in one window at a time? I'm worried about all the requests.

T

>115 timspalding: Oops, sorry. Is 15 tabs a bit much? :)

Yes. I saw you in the request list, um, more than once.

It is kinda hypnotic.

Glad I got to help -- sometimes projects like these are gone before I realize that anything is happening!

:-)

>106 staffordcastle: I remember that. It was very cool, like this cover size project.

>118 foggidawn:
Hypnotic indeed!

I went and dropped $50 at Amazon if it'll make them feel better.

This was fun. Now Tim: could you please find a bug in your code so we have to do it again :-)

121: I did, too (twice, actually ;-)

Yeah, I sent an Amazon order in late last night. Was that the point, really? Come on...you can tell us!

One of the few, the proud, the Maxed Out.

heh Even I didn't do 15 tabs at once. Only two. :)

+1 for the feather. Have metals, badges and feathers.

124>

Yeah, I sent an Amazon order in late last night. Was that the point, really?

I sincerely hope he wouldn't resort to such shady and underhanded tactics. I'm happy to help out LT, but very much not happy to help out Amazon.

>126 lorax: That would only happen during the rise of the AntiTim. For my part, I was just looking for any excuse to buy the Stieg Larsson books, which means the old world order is completely intact.

>126 lorax:

If you really think I spent hours crafting a new page in order to get 5% of two orders... :)

By the way, Amazon also has diamond jewelry and campers!

>AntiTim

We met. Collided. I'm what's left.

4> Ok, I'll help but only because I know you are taking a shower. Well, took a shower. I do not assume you have been in the shower for roughly 30 hours.

hmm. This might actually be interesting as a feature. Neat way of searching random books.

#127 And I was kidding...of course. I guess Tim didn't take offense, 'cause I'm still processing images, and I've gone over the 25,000 mark---or has that restriction been suspended now?

#128 Oh...hi there. And that was 3, maybe 4, orders, I believe.

I put in an order too, though that was actually because I really needed to make an order anyway ...

this is hypnotic! I love these completely random projects...

Shame that "That's enough for today!" doesn't take into account that "today" is now "tomorrow." Or yesterday. Or something.

Wow. Those covers really got to me.

LP, go look again - there's over a million left to go, Tim found some more to get!

I'm trying, but I'm in the "maxed-out" category.

Oh, then there's no hope - I still had a long way to go when we ran out last night, so I've had it running all day here at work :-)

Still haven't maxed out yet, but not much longer - I'm third from the top!

#79: It wouldn't surprise me if it's never been printed. The guy who does those has a script to rip Wikipedia and government sources and the like and dump it into a book, making adding a new book virtually free.

I'm running through a publisher that has apparently ripped off Google Books. I wouldn't mind the reuse of public domain material if they took the responsibility to at least check that all the pages scanned right. Instead they say "We believe this work is culturally important, and despite the imperfections, have elected to bring it back into print as part of our continuing commitment to the preservation of printed works worldwide", like they're actually making it more accessible.

> #138: Some people will say anything to justify their existence and the end product that they flog. I see too much of this kind of stuff in my line of work to be surprised anymore. I have adapted a habit of just rolling my eyes and getting on with my day. ;-)

I tried to help, but even with a broadband connection, my computer is just too old, slow, obsolete, lacking in memory, whatever . . . it froze and I had to do a manual shutdown.

I'll let those of you with more modern processors finish the job.

What happens when amazon arbitrarily changes images for a given ISBN?

Wow. I left it run while I went to work for a few hours. Now I'm ahead of Tim.

Just sayin'.

142> Tim is showering.

He'll be all wrinkly by now!

I've been running it for about a half hour now, and all I've been getting for the past 10 minutes are text-only covers. Nothing interesting at all. lol

>What happens when amazon arbitrarily changes images for a given ISBN?

A good point. In general, I don't think the image's aspect ratio will change very often. ISBNs relate to the printed matter inside, not to the cover. So you can change covers on a book and keep the ISBN. But if you reprint it with different paper you should get a new ISBN.

Should...

That's amazing! Tim has a keyboard in his shower so he can keep track of his beloved LT at all times.

I'm glad he doesn't have a live video feed though!!!

I'm glad he doesn't have a live video feed though!!!

What are you implying?

See #144!

Well, gosh-darn, I hit the lead. I'll max out inside an hour, I should think. If there's not a badge for this, a cheque (that's check for the USAians) made out to my ISP would also be accepted!

Only joshing, glad to be of assistance.

as of now it says there's about 180k left to do. wow. it was over a million yesterday.

What are you implying?

Nooo, no, nothing! You're a good looking man and all. I just prefer you in the abstract rather than the buff!

>152 bernsad:

Gods, I thought only Americans were so hung up about nudity!

wanders off in search of possibly shocking photos for profile image

*Makes note to keep checking Phaedra's profile*

And... it's back in the negatives.

We're now into negative numbers on covers left.

That's weird. How the heck does it get into negatives?

#157 Pick a number between four million and eight million. Perform an operation on some bunch of data fifty at a time in multiplexes of somewhere between ten and ten thousand. Admit that some of those multiplexes are going to fail before fulfilling their fifty operations. Subtract all those operations from the number you first thought of. Repeat until you get to exactly zero.

I don't know!

To make it fast, I had to randomize the picking such that, on average, 1/20 were done more than once. True randomization was too slow.

I'm calculating the "true" number now.

What I want to know is how we're getting covers, but when clicking on the link, Amazon says this is not a valid page on our site. Do they have the covers for books they don't sell?

Yeah, I suspect they're mostly non-Amazon US ones. (I don't get why they don't direct you to Amazon.de or whatever.)

Remaining: 528,540

?
The counter is in negative numbers - it looks like it was finished more than four hours ago!

Fixed.

Most of the books now aren't on Amazon in the US because we're past the English-language publisher ISBN zone.

LT is re-posting older reviews onto our walls at FB. Any connection? Or a different issue?

I was away for a long weekend and came back to this! I'm glad I didn't miss it. This is fascinating.

ETA: I'm getting all German books now. Wish I knew more German than "Schnell! Achtung!"

Ich liebe dich. Ich kann Glas essen, ohne mir weh zu tun.

Schnell! Achtung!

German soldier who can't make up his mind.

Gesundheit!!

I had a thought. What about the cover images uploaded by users and not from Amazon?

We can size those without your help... :)

What's different between Amazon covers and user covers, tho? That you can size those without our help? *wants to help more!*

Amazon comes from Amazon. User covers are those that have been uploaded by users. When you pull up a work and click on the Change Cover edit link along the left, you'll see the covers split into the 2 groups.

Perfect example would be the Harry Potter books:

http://www.librarything.com/work/3577382/covers/57868454

#173 - The images for the Amazon covers are on the Amazon site, so they had to be retrieved to get the size. The user covers are on the LT site so the size is available.

...and it's gone negative again. :)

And here I was, afraid I wouldn't be able to make 99,000 limit. :)

Just a little bit negative!

...and it's gone negative again. :)

No, not the negative numbers again! I just wanted to see what was at the end of the tunnel... Moving numbers move me too much.

We're in the negatives again: -2.598

173>

Tim needs member help because 8 million amazon cover requests--all generated from LibraryThing computers--would not only burden the servers, but would also attract the attention of the evil empire (not that we're doing anything in violation of amazon's policies or anything--but it's best to let sleeping dogs lie, no?). That's also why there is a cap to the number of covers processed by each user.

The dimensions for member-uploaded covers are either already known or can at least be determined in-house without querying amazon.

In short, LT is aware that there may come a day when it will be forced to declare its independence (with all due Jeffersonian rhetoric), but has no desire to hasten that day's arrival.

#178 Now my message makes no sense any more! ;-) You do know that little numbers become bigger, right‽ Or even smaller, in this case...

Right, smaller and smaller. (I was thinking about "just a little pregnant.")

GOT IT!

It was still crediting people with 50, not 25.

Actually left: 274,373

Tim, you've got yourself a feature here. I'll trade you timeline for it (see e-mail please).

Also, I'm adding this to my library. :-)

Woo-hoo! Negative again!

I just had to Kama Sutras in a row :D

>188 losseloth: You must be exhausted! ;)

Yup. Only 57,900 left. That's consistent with 1/20 random overlap, with four people doing it now.

Getting very close.

So are we supposed to keep it open when it says negative? *confused*

Yeah. It's lying to you.

33,736 left!

#188 two Kama Sutras in a row
#190 four people doing it now

Heh.

194: *giggle-snorts*

It's negative numbers AND not showing any covers. Meep.

Maxed out before it finished -- too bad!

Well, all I have to say is "thanks." Rise and repeat that 6 million times.

We've going live with this feature tomorrow. Members won't see it, but the intended users will, and it will go live to all members on Monday. Absent cover sizes, it wouldn't have worked.

Which it is my pleasure to be of assistance.

ps How many covers did we end up doing?

6 million?

Plus another "two million or so" from #107.