Talk doesn't re-escape HTML entities on editing

(Reposting for new bug feature)

When you post a message to talk, you must escape certain html entities (e.g. < with &lt;). But when you edit a post, LT does not re-escape the entity. So if you click edit and make changes and click save, these escapings are lost.

(Note - I've heard some people actually try to classify this as a "feature" or just "incomplete functionality". That's pretty silly, in my opinion.)

brightcopy, when you're re-posting bugs, it's very helpful to supply a link to the previous discussion thread, so Tim can get the rest of the discussion!

Actually, I wasn't reposting this from any particular thread. It's an oooooold thing that's been tossed about in numerous threads, and one Tim is already well aware of.

A quick aside since we're on the subject: What tags are we allowed here?

If you don't mind, please repost that as a different thread in Talk about Librarything. This is a bug report. :)

I don't care if you think it's silly to call it a "lack of feature" rather than a bug, but that's what it is -- the Talk feature parses HTML, and doesn't choose to keep around the original format on the off-chance that it will be edited at some point in the future. It would make a good RSI, but it's not a bug.

Yeah. And if the Talk edit feature decided to cut off the last half of your message when you hit Edit and Save, that'd be a similar "lack of feature."

ETA: And FYI, your "didn't keep around the original format on the off-chance that it will be edited at some point" makes no sense. It keeps the original format. If not, how do you think it know to display these two things separately to you right now?
<a href="http://www.librarything.com/">This is a link</a>
This is a link

7>

No, the cutting off would be a bug -- something that didn't work as intended. This is a lack-of-feature -- something that doesn't work optimally, or not how you want it to.

It *parses* the original format into the code it displays. It does not *keep* the original format afterwards. It doesn't need to keep it after parsing in order to display it correctly.

8> Honestly, lorax, I really think you don't know what you're talking about here, in regards to how it parses, stores and displays it. It's a very basic data storage task. If it didn't keep the original format, it would not know how to display it.

If you view the source code for this page, the above snippet looks like this (with escaping done by me so you can see it in Talk. but if you just do a view page source, you'll see this):

If not, how do you think it know to display these two things separately to you right now?<br>&lt;a href="http://www.librarything.com/"&gt;This is a link&lt;/a&gt;<br><a href="http://www.librarything.com/" rel="nofollow" target="_top">This is a link</a>

If you can't figure out what I'm talking about with this, I give up on trying to explain it to you.

I'll admit I hadn't viewed source on LT, and that you're right. If you hadn't been so condescending at first with the "how else would it know the difference", I probably would have admitted that earlier.

10> Ah, I see, everything is MY fault. Got it. That would be consistent today.

ETA: And for the record, I wasn't being condescending by asking you how else it would know the question. It was a legitimate question to make you think about it, since you hadn't already before you jumped in. You read condescension into it. For comparison, I'm most definitely being condescending now.

This is a test of bold. This is a test of how to do bold: <b>

And this is a touchstone for Every Visible Thing and Lisa Carey.

Okay, fixed. I'm worried problems will crop up. But it's fixed for now.

You're my hero!

Err... just ignore the "boos"...

Yay!

Tim I think you broke something. It seems to be trying to 'unescape' apostrophes. While keeping the HTML is great this other one is a bigger pain.

I don't seem to be getting the same issue. Which "apostrophe" do you mean? Do you mean this '
Or this `
Or this "

Or some other one. I know a regular apostrophe is ', but I'm not having the issue with it so I'm just making sure there isn't a communication gap (e.g. slash vs backslash).

ETA: Yeah, just edited and none of those are unescaping.

Yes, someone give me a concrete example. If it's to a post in the past, give me the number and I'll log in as you and try to edit it too.

http://www.librarything.com/topic/99634

when I try to edit either of my posts 1 or 3 I get those little diamond shapes.

19> I went up to the first post in this thread and clicked edit, and it also didn't do anything weird as far as I could see.

So far I've:

1. Clicked edit on an old post, adding an apostrophe, saved, clicked edit again
2. Clicked edit on an old post that already has an apostrophe
3. Created a brand new post with apostrophes, clicked saved, clicked edit again

None of those has exhibited anything other than normal behavior, as far as I can see.

Got it. It's a UTF8/Latin-1 problem. I think I know how to fix it. I'll look into it.

20> Ahhh, as I thought, those aren't the typical apostrophes used. They're arguably the CORRECT "apostrophe", though.

Here it is:
’

20>

You've got strange non-standard apostrophes there, that are actually some non-ASCII character -- did you copy and paste from somewhere that uses "smart quotes"? I can see how that would screw things up.

24 > Yes, I copied it from Google docs and I was coming to that conclusion (not normal apostrophes) since no one else seemed to be having the same problem.

24> Good catch, though.

I believe this is the same bug, but just in case:

http://www.librarything.com/topic/99743

Would you rather mark this bug as Fixed and use that one as the official bug for this variant?

A testing of the apostrophe...

’

And now bold and <b>!

Testing a lot of stuff:

é á í ü ö à è ñ ã î ô

ETA: Nope, still boned. This was originally:
é á í ü ö à è ñ ã î ô

Fixed. Sorry. See #28. Both entity quoting and charsets now working on submission and reedit.

So in theory this works now. Testing:

é á í ü ö à è ñ ã î ô

Woohoo! Looks good.

For documentation purpuse it should be possible to generate metacharacters.

At http://www.librarything.com/topic/99305#2254475 I intended to generate LEFT SQUARE BRACKET (U+005B) and RIGHT SQUARE BRACKET (U+005D) which are LT talk metacharacters used for Touchstone syntax.
&+#x5b; and &+#x5d; will not show up. Is there another way to generate them as normal characters?
As far as I rememeber it was possible to generate these characters before the fix of this bug; they did not interfere with Touchstone syntax.

Hmmm... looks like brackets are broken. Creating new bug.

ETA:
http://www.librarything.com/topic/100730