Pics/Covers Technical Issues

>453 Cover and Pics Issues
Quick note: I've disabled the loading, display, and picking of many covers on the Work: Change Cover pages.

THIS IS TEMPORARY! Hopefully just hours.

Want to know why? Read on.

LT is, in general, having a real time with our entire pics/cover infrastructure this week. We have most/all of our technical staff working on it to some degree. The failures, like many failures that happen to complex systems, have been at multiple levels and created a kind of perfect storm.

Some of the failures have been purely technical: our caching level that is supposed to insulate us against the load of hundreds of thousands of images a day is, for some reason, failing to do so. We are investigating.

Our own pics servers (a cluster of them) were simultaneously not serving content as quickly as they should have. This caused a kind of domino effect where they continued to get slower as requests piled up and they couldn't serve them quickly enough. This was, of course, itself a domino effect from our caching layer not working, as noted above.

Added to this was a terrible mistake by me when designing the way the new cover pages and a couple of other elements (even on some older pages) load their content. Long story short, I miscalculated how the browsers would optimize the inline loading of images on these elements at the network level. That caused a higher than normal number of requests for images on the these elements, especially the Change Cover pages.

This is also why you've been experiencing slow loading times on the cover pages. Of course, this higher than normal load then creates an added domino to the first two issues above...and that's how you get a catastrophic failure situation: multiple things must go wrong at once.

WE ARE WORKING ON THIS from multiple angles and at multiple levels.

May I suggest a banner directing people to this thread? Some folks may not look at the forums, or this group, and freak out!

>2 lilithcat: Thanks. There is a banner/alert in the spot where we are limiting the cover selection. It has a message that points to this thread.

ETA: and I've added a site-wide banner that should show up soon.

>3 conceptDawg: Thank you, info by banner is a great idea!

>3 conceptDawg: Thanks!

Agreed with all those who say that site-wide banners are a great idea when something this important is going on.

I've enabled the display of all of the member covers again on Work: Change Cover pages. That page is now much more efficiently loaded. We are still battling the server-level infrastructure issues.

Thank you very much for the banner, it is very helpful!

Thanks for the banner notice, I had been wondering what was going on. Appreciate all your efforts for the LT site :)

I mentioned in a different thread a couple of weeks ago, that photo uploads to profiles seem to have a stutter. Images are loading multiple times,, possibly without the member noticing.
See this gallery for example
https://www.librarything.com/gallery/member/EricBerman

Thank you for keeping us updated

Is this why the Add Books Manually is now a shadow of its former self?

>12 joedoone:

Can you explain what you don't like about the manual-add page now?

Re the Manual add function: for the past two days, the Media Type and the Language fields do not save consistently. There may be others, but these are the ones I've encountered. Occasionally it does work on the first save attempt, mostly yesterday it was taking two or three saves to make it retain the info in those fields.

Is something wrong with the Recommendations section? All-Time, even limited to a specific genre, used to list up to 14 pages of recommendations for me. Now there is only one page.

>14 TJLegacy4U: Re the Manual add function: for the past two days, the Media Type and the Language fields do not save consistently. There may be others, but these are the ones I've encountered. Occasionally it does work on the first save attempt, mostly yesterday it was taking two or three saves to make it retain the info in those fields.

Is this when you add a new book or when you edit an existing one?

>16 knerd.knitter: When adding a new book. Right now, the Primary Language field is saving, but the Media field defaults to "Not set" after saving the new book record. On the second try, when going through Edit, it did save.

Obrigada pelo aviso!

The updates and banner are much appreciated!

I hope you've got some nice soup or cheese or pie, or whatever you prefer, to get a little break and mental boost when lunch time comes around.

With all this mess, did you all include cold weather gear for when the two big snow storms flow through this week/weekend?

Noticed there was some problem a couple days ago when I thought I'd clean up my photos, and found my gallery simply wouldn't load everything. Glad it's under management!

>17 TJLegacy4U: Good catch! We narrowed this down. I'm finding this only happens on the initial manual add, when you also add a Media type, using the complete list (not the short list).

Thanks for the notice & the hard work. You guys are appreciated

Thank you for the information.

Many thanks to the LT team for the banner, update and patience in replying to so many queries. I have noticed that since the new system went up, sometimes the covers do not load, not with newly added books nor ones that were already logged. More often than not when going to "Your books", images either did not load or appeared on top of one another. Hopefully this will sorted in the next couple of weeks, on Chrome and Safari.

>25 LLoLaguayasaminaj: Hopefully much sooner than that. Or I'm going to need to do some serious soul searching.

Thanks for the update - and best wishes. That cascade of problems....Dang!!

>17 TJLegacy4U: This issue should now be fixed! Thanks for the report.

>22 kristilabrie: Excellent! Thanks for all your good work.

Update: pics/images
We think we have found the underlying problem and have rolled out a change to our server infrastructure.

You should be seeing much better image loading across the site. Many of them won't load instantly like they should but that's because we're temporarily running with fewer image servers than normal because we had to take some of them offline that were causing the problems. BUT, all images should load now and we are seeing error rates near zero.

We'll try and get everything back up to full capacity by the end of the day.

>30 conceptDawg: Thanks. Appreciate the banner message alerting about the issue though on the homepage.

>1 conceptDawg: Thank you so much for taking the time to provide explanations - and having that breed of courage so rare in todays world: the courage to admit having made a mistake.
We all make them. I'm astonished when I consider the vast number of variables at play in a system this complex that it WORKS AT ALL. Kudos to you and all the other staff for this stupendous work of love.

Thanks to all LT staff to respond quickly when a problem occurs. You guys are doing a good job.

The banner is much appreciated. That way I know that's somethign wrong before I hit it.

I know you're still working on stuff, but the covers are loading much better already! Thanks! And getting the banner up so quickly is amazing work, too.

I can confirm that all the covers that didn't load yesterday seem to be loading today, at least based on a brief check.

Thank you.

Thank you for this update. Your courage to admit that mistake was made is commendable. Looking forward to a quick resolution by your team.

Thank you for keeping us posted.

Very much appreciate your candor and efforts. Let me know if there's anything I can help with.

>1 conceptDawg: Thank you so much for this update!

And how can I permanently dismiss the banner, please?
It’s a persistent bugger.

The banner should now be gone. >41 2wonderY: We don't use the banners very often so it's possible there was a problem with the dismiss action on this one. I'll check it before we use the banner again (hopefully won't need it soon).

>42 conceptDawg:

The banner did not appear possible to dismiss, no.

>42 conceptDawg: Thanks. It’s gone now.

Last night (PST) I had slow image loading.

Around noon (PST) today I had occasion to view another cover image. It was still extremely slow. I thought of the days when I had a 28.8 kbps modem (but not the days of 300 baud, thankfully).

I looked for a post on the topic last night but did not see one. Today, when it was still an issue, I found this one. Based on my limited testing, this issue persists.

It does seem improved to perhaps an acceptable level. I hope it will return to normal load times soon.

In my testing (looking at detail pages, cover view after a search, etc.), I am trying to work with items I have not viewed recently so they won't be in a local browser or server cache. I am seeing come covers that take 15-20 seconds to load for a file that are smaller than 1 MB. One was 557 kB, for example.

I will repeat that it does seem to be improving. Perhaps servers have been restarted and caches are being rebuilt. I do some of this with my day job (we use Varnish for a few thousand WordPress sites) so I understand some of the challenges you are facing.

James

>14 TJLegacy4U: Thanks for noticing this. I had one book where the language fields disappeared when I edited the review, but I was late in finding out and couldn't reproduce the error.

Was the banner posted? I've not seen the banner the last two hunts and I'm not seeing this one, either. I'm hoping that will be fixed for me (and others who don't see it) at some point, as well.

As for this topic, yeah... I wondered about the slow loading covers when I went to change one this evening.

>47 LibraryCin:

It was, but it’s since been removed. See >42 conceptDawg:

The covers are back to not really working this morning, is this just me, or are we back to the drawing board? (I am prepared to be patient.)

Edit: And now that I've said that, it appears to be better. I have no idea what I'm doing.

>49 AranelST:

No, I'm still seeing some trouble too. Slow loading, and sometimes broken cover images. Refreshing the page fixes the broken ones. So, nothing fatal, but definitely not full health.

We're still going to have some intermittent problems until we can get another server up and running. But we have staff actively working on the problem.

>29 TJLegacy4U: That was all @knerd.knitter's doing! :)

I've noticed some odd behavior around cover thumbnails in the "Change Cover" view on the new Work page. Example: on Golden Comics Digest #25: Tom and Jerry Special Stories, after selecting Change Cover, I see three images, all sourced from a cover I uploaded this afternoon. From left to right (I'd post a screencap if I could, sorry), there's 1) the cover I've selected for my copy, 2) the thumbnail for that same cover in the Change Cover > Potential Covers view, and 3) the thumbnail for that same cover again under Popular Covers. What's odd is that for 1 and 3 the page is retrieving the 200px version of the thumbnail and having the browser scale it down for display, but for 2 it's retrieving the 100px version and having the browser scale it *up*. Is this intentional? How does LT decide whether to use a 100px thumbnail or a 200px thumbnail for a potential cover when (as here) both sizes are available?

I have noticed a few of my records are showing covers rotated 90 degrees from what they should be
pic ID: 4796826 https://www.librarything.com/work/214923
14392368 https://www.librarything.com/work/167074
14392373 https://www.librarything.com/work/52654
14392389 https://www.librarything.com/work/52922
14392386 https://www.librarything.com/work/52930/book/19353122
14392380 https://www.librarything.com/work/52792/book/19353129

This may be resolving itself as a result of the other fixes to images? I think there were more instances yesterday. And my cover for https://www.librarything.com/work/167163 shows as rotated in my catalogue, but the work page has all covers oriented correctly

>48 lilithcat: Thank you. I missed that it got removed, so maybe I just missed seeing it while it was up.

>53 LazloNibble: Sorry, it's hard to tell what you're describing here.
But we define the various sizes that are available to display for that same image. The browser should display only one of those, based on heuristics that it determines for your browser, display, network bandwidth, etc.
Now, you should only be seeing ONE of those images. And, in fact, I only see one of them when I view that page. So it's hard to tell exactly what you're seeing.
A screenshot in this case would be invaluable if you could get one and upload it to your LT profile gallery (or Junk Drawer even).

>54 Avron: yeah. That's a long-standing issue with our image resizing library that we use. It's not reading the EXIF metadata from the images correctly in some cases.
I REALLY want to fix that, so having these as samples is nice to have.

>57 conceptDawg: Here's another example for you https://www.librarything.com/work/19014819/148327875

I uploaded a new image today when I saw the rotated cover and the newly uploaded image is showing up in portrait, but the previous (rotated) cover is still there.

Is flagging covers disabled? Looking specifically at The Omnivore’s Dilemma: Young Reader’s Edition …

No. Click on the magnifying glass in the corner of the banner, then click on Flag Cover. Still there. I flagged two covers this morning.

>56 conceptDawg: Thanks, I've posted it to my Junk Drawer - https://www.librarything.com/pic/11451999. This is as I saw it originally. As of right now, I'm getting the correct behavior in that all three images on that page are the 200px version. A direct comparison of what I saw previously vs. now: https://www.librarything.com/pic/11452027

Not sure whether this issue ever got "resolved" . . . a month later I'm still having trouble seeing covers.
Today I was trying to check the covers for several newly added books.
When I go to "change cover" and then look at the Member loaded covers (the only ones I'm interested in using) and click on the little magnifying glass to see the details,
*no image shows in the pop up box
*and often no details about the size of the image, etc.

I'm also missing the dimension info that used to appear in the screen with all the choices . . . is that no longer going to be included??

Will put this project off yet another month, but could use an update on what's going on in covers. (I checked in Bug Collectors but didn't see anything matching this particular problem and don't want to post as a bug if you're still working on this issue.)

Thanks for any update you can provide.

Same. Meant to post or ask about it, but forgot. It's been a problem for a few days - covers not showing in the catalog properly or in the recently uploaded covers module on the home page.

>62 LucindaLibri: >63 Bookmarque: Please see https://www.librarything.com/topic/368933#n8778202