Issue Details (XML | Word | Printable)

Key: MBS-3680
Type: Bug Bug
Status: Decision Required Decision Required
Priority: Normal Normal
Assignee: Unassigned
Reporter: Brian Schweitzer
Votes: 3
Watchers: 5
Operations

If you were logged in you would be able to see more operations.
MusicBrainz Server

Error 502 Bad Gateway loading certain releases from the webservice

Created: 22/Oct/11 06:08 PM   Updated: 25/Apr/13 04:58 PM
Component/s: Web service
Affects Version/s: None
Fix Version/s: None

Issue Links:
Duplicate
 
Relates
 


 Description  « Hide

I'm seeing a 502 Bad Gateway error persistently loading certain releases into Picard. The releases appear valid, and if I strip out random bits of the includes in these requests in a browser, then no error is received.



Sort Order: Ascending order - Click to sort in descending order
voiceinsideyou added a comment - 23/Oct/11 04:02 AM

My guess is that this might be related to some of the issues I raised in MBS-3122 (which unfortunately is not very actionable in current state); in particular work-level ARs going back to OTHER recordings than the ones on the release.

My guess is that if you remove work-level-rels from those; they will probably all work.


Brian Schweitzer added a comment - 23/Oct/11 12:00 PM - edited

The behaviour certainly seems the same, at least in that those particular requests take a good while before the server decides to return the error. It takes maybe 10 - 15 minutes for Picard to finally finish [could not load foo-foo-foo-foo]-ing all of the above.

Oddly, as I said, if you take out any bits of the request, chosen at random, the request works: http://musicbrainz.org/ws/2/release/f5378fe3-f999-4357-9c3f-6cb0ca0c8bbd?inc=work-level-rels for example, or http://musicbrainz.org/ws/2/release/f5378fe3-f999-4357-9c3f-6cb0ca0c8bbd?inc=release-groups+media+recordings+puids+artist-credits+artists+aliases+work-rels+recording-level-rels+work-level-rels+tags - both work, but http://musicbrainz.org/ws/2/release/f5378fe3-f999-4357-9c3f-6cb0ca0c8bbd?inc=release-groups+media+recordings+puids+artist-credits+artists+aliases+labels+isrcs+artist-rels+release-rels+url-rels+recording-rels+work-rels+recording-level-rels+work-level-rels+tags does not.

Side note: mentioned in MBS-3122 is the size of the returned data... all those repeated aliases for Bach really seems redundant and bloaty. I guess the secondary artist data couldn't be moved out to a second single top-level <artist> at the <release> level, and cut all those out?


Brian Schweitzer added a comment - 04/Nov/11 04:58 AM

More of these problematic ones:


Clemens Heidinger added a comment - 05/Nov/11 01:04 PM

Another release that won't load in Picard: musicbrainz.org/release/9c5c043e-bc69-4edb-81a4-1aaf9c81e6dc
It is a huge release and has 940 tracks for which each of its recordings has several ARs attached. If "Use track relationships" is disabled in Picard, the release will load.
If the option is enabled, the web service call of http://musicbrainz.org/ws/2/release/9c5c043e-bc69-4edb-81a4-1aaf9c81e6dc?inc=release-groups+media+recordings+puids+artist-credits+artists+aliases+labels+isrcs+artist-rels+release-rels+url-rels+recording-rels+work-rels+recording-level-rels+work-level-rels fails with error code 502 which causes Picard to not being able to load the release.


voiceinsideyou added a comment - 05/Nov/11 03:00 PM

I think the easiest way to fix this in a general sense will be to change the way work-level-rels works; to not return links back to other recordings (or at least provide this as an option). I'd planned to spend some time thinking about it in more detail to propose a solution for Picard's purposes, but haven't really got around to it


voiceinsideyou added a comment - 02/Dec/11 12:32 PM

See my comments on this on the mailing list at http://lists.musicbrainz.org/pipermail/musicbrainz-devel/2011-November/004579.html

I'm away for a few weeks and have not had time to log separate related tickets on actions that we can perform to mitigate this. Only short-term workaround is to turn off track relationships in Picard.



Oliver Charles added a comment - 28/Dec/11 04:45 PM

Seems to be the same kind of problem as MBS-3841?


Oliver Charles added a comment - 27/Jan/12 01:54 PM

Faster XML serialization is in review at http://codereview.musicbrainz.org/r/1755/ which should get these going through in under 30seconds now.


Oliver Charles added a comment - 07/Feb/12 01:11 PM

All URLS from the description respond with a 200 now. They still take up to a minute to fully produce a response, but they don't time out.



Paul Taylor added a comment - 07/Feb/12 05:29 PM

Confirmed that if you remove work-level-rels it works

http://musicbrainz.org/ws/2/release/0b86e3b2-c78e-4691-b3f1-2e003f0c5f2c?inc=release-groups+media+recordings+puids+artist-credits+artists+aliases+labels+isrcs+artist-rels+release-rels+url-rels+recording-rels+work-rels+recording-level-rels+tags

I think there are a number of things we could consider:
1. Extend timeout to allow these superlarge queries to work, on the basis that the impact is not too large because they are few and far between.
2. Precache these particular lookups, so that the data is already formed and can be returned immediately.
3. Reconsider work-level-rels, these weren't in the original release of NGS and perhaps they are too low in the hierachy for them to be retrievable.
4. Extend Webservice so that if there is too much data, it only returns some and provide some kind of paging mechanism to get the remainder.


Oliver Charles added a comment - 08/Feb/12 11:14 AM

I personally think 1 is the only really viable option right now. I'm curious about 2 for sure, but that'll need some more thought. 3 and 4 break the web service as far as I'm concerned, and I would only like to look at them as a very last resort.


Oliver Charles added a comment - 08/Feb/12 11:16 AM

Note that there's another optimization in review atm, which might help this: http://codereview.musicbrainz.org/r/1738/


Paul Taylor added a comment - 08/Feb/12 08:07 PM

The thing about work-level-incs is that they were added later maybe without considering the effect they had so it wouldn't be unreasonable to take another look at them and not remove them but trim the extra information they output, in a earlier comment voiceinsideyou seems to be coming to the same conclusion, and I always consider voice to be be the voice of the people.

The last optimization you mention isn't going to have much impact, we need a solution that deals with the queries that return drastically more information then the norm rathert tahn just shaving some time of all queries (although of course that is also welcome)


Oliver Charles added a comment - 13/Aug/12 11:25 AM

We're blocked on this again, so I'm moving it back to decision required to get it out of my work queue. It's scheduled for discussion in todays agenda.