Reconsider README being included in the full text search index

Searching for most common words in https://crates.io will usually return a lot of results, e.g. https://crates.io/search?q=foo. That's because of README.md being full text indexed.

I think it's useful to have README.md indexed but perhaps it could use some preprocessing such as dropping code blocks to reduce the amount of noise.

[alexcrichton]

Postgres has support for ranking different parts of a document differently for searching, and the README is definitely the lowest factor on the spectrum. I figure it's better to give at least some tangentially relevant search results at least, but this would be very bad if README-hits were higher up than a crate called foo, for example.

Do you have some sample searches which go awry? Or were you just trying to reduce the amount of results?

Yeah, this issue borders on a bike shed but I thought it was strange to see so many results for foo that were there only due to a variable being named foo in a code block in their corresponding README.md. But I can imagine situations where that's actually useful so this may as well not be a bug at all.

[alexcrichton]

Would you be ok closing this in favor of #81? If we showed it on the page it may be more obvious why it's being indexed.

Certainly, cheers!