|
Sort Order:
I also think url is pointless. I've been thinking about /url/ since the meeting, and especially since we cannot search through URLs via MusicBrainz, I think this might actually be useful. Ie., to have them unmasked. It may be possible (once MusicBrainz has been indexed with the new paths) to e.g. search with site:musicbrainz.org/url/. You don't need to open up /url/ to do that. URLs are always linked to an entity, so will (if recordings are indexed) always be part of another indexed page. e.g. http://www.google.de/search?q=site%3Amusicbrainz.org+http%3A%2F%2Fwww.youtube.com%2Fmorningmusumechannel I think I'd rather see 1.5 million more interesting pages indexed... In the long term I'd like /url to be opened up, because I still haven't seen a convincing reason otherwise. People are making all sorts of assumptions about search indexers - such as if we open /url it will suddenly decide to index nothing but /url, but I highly doubt that is the case. However, I can't back up my refutal of that claim, so I'm fine with just opening /release and /work for now, and /url can come later. I'm not claiming it will suddenly decide to index nothing but /url/ any more than you're claiming it will not do any /url/ pages until it's done everything else. It just seems logical that if it spends some of its time indexing /url/, that's time it's not spending indexing pages with more interesting content.
I know, that's why I said "I can't back my claim up either" and suggested we continue without /url. nikki said: This is a great argument. With that in mind, lets leave /url off for now. In a few months, lets revisit this and see if we want to do this. I updated the pull request. |
||||||||||||||||||||||||||||||||||
Task
Closed
Normal
Open up /release/ and /work/ in robots.txt
https://github.com/metabrainz/musicbrainz-server/pull/10