Show HN: Convert HTML to markdown with python (opens in new tab)

(github.com)

29 pointsgaojiuli8y ago14 comments

14 comments

You should use HTML.parser and focus on the conversion to markdown. The way you parse HTML in the convert function is very inefficient and can easily produce incorrect results with valid HTML (e.g. stuff">some text )

williamstein8y ago

Since this is "Show HN", what is the motivation you have for doing this? There's no motivation or background in the linked README.

gaojiuliOP8y ago

When crawling online articles such as news, blogs, etc. I want to save them in markdown files but not databases.

Bino8y ago

Wouldn't it make sense to escape markdown syntax in the HTML? In HTML which you're converting from. * has not special meaning.

foo

gaojiuliOP8y ago

Tomd has the ability of converting the HTML that converted from markdown. If a HTML can't be described by markdown, tomd can't convert it.

jwilk8y ago

How is it better than https://alir3z4.github.io/html2text/ ?

gaojiuliOP8y ago

Maybe the number of lines of code is less. And the logic of tomd is easier to understand.

roryisok8y ago

Nice to see projects that convert HTML to markdown. Projects to convert in the other direction (md to html) are far more common. Parsing HTML is tough

4e1a8y ago

I completely agree. I had made something similar in lua and it was very fun and I found many md to html converters, but not the other way around. HTML parsing is what made me 'pause' development on that tool.

confounded8y ago

Pandoc may be of interest.

gaojiuliOP8y ago

Tomd is a python tool. Pandoc is a haskell tool.

cyphar8y ago

Yes. Though pandoc handles more formats and doesn't parse HTML using regex. And it supports more features.

In general the language something is written in isn't necessarily a benefit by itself. Is there a reason why you prefer Python when there already exists another tool written in another language?

1 more reply

j / k navigate · click thread line to collapse

14 comments

krstf138y ago

williamstein8y ago

Since this is "Show HN", what is the motivation you have for doing this? There's no motivation or background in the linked README.

gaojiuliOP8y ago

When crawling online articles such as news, blogs, etc. I want to save them in markdown files but not databases.

Bino8y ago

Wouldn't it make sense to escape markdown syntax in the HTML? In HTML which you're converting from. * has not special meaning.

foo

gaojiuliOP8y ago

Tomd has the ability of converting the HTML that converted from markdown. If a HTML can't be described by markdown, tomd can't convert it.

jwilk8y ago

How is it better than https://alir3z4.github.io/html2text/ ?

gaojiuliOP8y ago

Maybe the number of lines of code is less. And the logic of tomd is easier to understand.

roryisok8y ago

Nice to see projects that convert HTML to markdown. Projects to convert in the other direction (md to html) are far more common. Parsing HTML is tough

4e1a8y ago

confounded8y ago

Pandoc may be of interest.

gaojiuliOP8y ago

Tomd is a python tool. Pandoc is a haskell tool.

cyphar8y ago

Yes. Though pandoc handles more formats and doesn't parse HTML using regex. And it supports more features.

In general the language something is written in isn't necessarily a benefit by itself. Is there a reason why you prefer Python when there already exists another tool written in another language?

1 more reply

j / k navigate · click thread line to collapse