Post History

Python BeautifulSoup how to correctly parse quote symbols in link titles (broken html)?

I'm learning some basic web scraping so I can download part of a static website for offline viewing. In the process, I'm running into link tags that look something like this:

```
<a href='https://example.com' title='Example web author's link to another page'>Page X</a>
```

Or, that's what I'd expect to get when looking at the page source. When I run `html = urlopen(url).read()` and then `soup = BeautifulSoup(html, 'lxml')`, BeautifulSoup somehow mangles the link title so it ends up looking more like this, depending on the text and number of quotes:

```
<a href="https://example.com" s'=" link to another page" title="Example web author">Page X</a>
<a page'="" href="https://example.com" s="" title="Author">Page X</a>
```

I'm certain the odd number of quote symbols is throwing the thing off by making it think the title ends earlier than it really does. It's a bit of a problem since `link.get('title')` only returns the part of the title before any quote symbols. Changing the parser didn't work and search engines are only giving me generic BeautifulSoup articles when I look this up, so I've no idea how to fix it. I'm running BeautifulSoup version 4.14.2 on Arch Linux if that helps.