|
Sort Order:
Putting a rough scale estimate as city on this. The reports are currently ran once, so we'd need to add in some extra logic to do the filtering. Also, it's not clear how each report should actually be filtered, so each report may need different logic. I guess the reports have to be filtered "at runtime" somehow, it is of course not possible to generate a different report for every user. Currently, I filter the reports myself (on my slave server) with a huge list of auto-generated SQL statements. Since subscriptions are currently only possible for artists (not e.g. releases), the logic is quite similar for all the reports. Of course, if we allow subscriptions to releases, this is not clear anymore. I'm working on this... currently I have a version running that filters by artist if "?subscribed_artists=1" is appended to a report. It needs to be polished (UI, code), than I will put it up for review. How are we looking on this hrglgrmpf? I have a prototype that can filter all simple release reports. I'm still undecided how to handle MBS-2919: Either create a separate table (which reports are filtered/unfiltered), or store it in the same table as the report. I think I like the second option more, but it means a simple DROP/CREATE is not possible anymore, as the state of the flag must be preserved. What do you think? MBS-2919 can come later, for now filtering by subscriptions is more important. Let's deal with that one when we get to it. Is this ticket ready for review? Any more news on this? Due yesterday apparently I'm afraid the prototype isn't ready for code review yet hrglgrmpf has said he'll need a bit more time for this, so I've pushed the due date back 3 months from the original date of February. It's almost a month past the due date. Any update? Is that it, just assigning to no-one without any explanation makes it difficult for anyone considering taking on the work to know where to start form. I will take this up. @Luca - it's going to slip a little more behind I'm afraid, but I will hopefully have something working by the end of July. No promises there though! I'm very sorry about the delay... I started working recently and have less time than I thought to work on such a big ticket, so I'm unassigning myself. My progress so far was that I migrated some reports to a SQL table, which is then filterable on the fly. It was not that much work, but it was also quite hackish, so a clean solution (which covers all reports) needs a lot more work. Here are some of the challenges that I found, for the next person to work on the ticket:
I hope I could help, and the ticket will be solved soon (despite me unintentionally delaying it for several months...). This is now in code review at http://codereview.musicbrainz.org/r/1985/ I should note that it's currently running reports on hobbes, so reports won't work just yet. On http://test.musicbrainz.org/report/ISRCsWithManyRecordings Internal Server Error
Oops, something went wrong!
Error:
Caught exception in MusicBrainz::Server::Controller::Report->show "Failed query:
'SELECT DISTINCT report.* FROM report.isrc_with_many_recordings report ORDER BY row_number OFFSET ?'
(0)
DBD::Pg::st execute failed: ERROR: relation "report.isrc_with_many_recordings" does not exist
LINE 1: SELECT DISTINCT report.* FROM report.isrc_with_many_recordin...
^ at lib/Sql.pm line 79.
at lib/Sql.pm line 84
Sql::__ANON__('DBD::Pg::st execute failed: ERROR: relation "report.isrc_wit...') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Try/Tiny.pm line 100
Try::Tiny::try('CODE(0xc3843a8)', 'Try::Tiny::Catch=REF(0xc3becd8)') called at lib/Sql.pm line 86
Sql::select('Sql=HASH(0xba82620)', 'SELECT DISTINCT report.* FROM report.isrc_with_many_recording...', 0) called at lib/MusicBrainz/Server/Data/Utils.pm line 224
MusicBrainz::Server::Data::Utils::query_to_list_limited('Sql=HASH(0xba82620)', 0, 50, 'CODE(0x5f85a30)', 'SELECT DISTINCT report.* FROM report.isrc_with_many_recording...', 0) called at lib/MusicBrainz/Server/Report.pm line 47
MusicBrainz::Server::Report::_load('MusicBrainz::Server::Report::ISRCsWithManyRecordings=HASH(0xc...', '', 0, 50) called at lib/MusicBrainz/Server/Report.pm line 30
MusicBrainz::Server::Report::load('MusicBrainz::Server::Report::ISRCsWithManyRecordings=HASH(0xc...', 50, 0) called at lib/MusicBrainz/Server/Controller/Report.pm line 38
MusicBrainz::Server::Controller::Report::__ANON__(50, 0) called at lib/MusicBrainz/Server/Controller.pm line 191
MusicBrainz::Server::Controller::_load_paged('MusicBrainz::Server::Controller::Report=HASH(0xacf5490)', 'MusicBrainz::Server=HASH(0xc203560)', 'CODE(0xc3be690)') called at lib/MusicBrainz/Server/Controller/Report.pm line 40
MusicBrainz::Server::Controller::Report::show('MusicBrainz::Server::Controller::Report=HASH(0xacf5490)', 'MusicBrainz::Server=HASH(0xc203560)', 'ISRCsWithManyRecordings') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Action.pm line 65
Catalyst::Action::execute('Catalyst::Action=HASH(0xb194890)', 'MusicBrainz::Server::Controller::Report=HASH(0xacf5490)', 'MusicBrainz::Server=HASH(0xc203560)', 'ISRCsWithManyRecordings') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1667
eval {...} called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1667
Catalyst::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb194890)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Plugin/StackTrace.pm line 94
Catalyst::Plugin::StackTrace::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb194890)') called at lib/MusicBrainz/Server.pm line 306
MusicBrainz::Server::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb194890)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Action.pm line 60
Catalyst::Action::dispatch('Catalyst::Action=HASH(0xb194890)', 'MusicBrainz::Server=HASH(0xc203560)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Controller.pm line 125
Catalyst::Controller::_ACTION('MusicBrainz::Server::Controller::Report=HASH(0xacf5490)', 'MusicBrainz::Server=HASH(0xc203560)', 'ISRCsWithManyRecordings') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Action.pm line 65
Catalyst::Action::execute('Catalyst::Action=HASH(0xb1944a0)', 'MusicBrainz::Server::Controller::Report=HASH(0xacf5490)', 'MusicBrainz::Server=HASH(0xc203560)', 'ISRCsWithManyRecordings') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1667
eval {...} called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1667
Catalyst::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb1944a0)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Plugin/StackTrace.pm line 94
Catalyst::Plugin::StackTrace::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb1944a0)') called at lib/MusicBrainz/Server.pm line 306
MusicBrainz::Server::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb1944a0)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Action.pm line 60
Catalyst::Action::dispatch('Catalyst::Action=HASH(0xb1944a0)', 'MusicBrainz::Server=HASH(0xc203560)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Dispatcher.pm line 257
Catalyst::Dispatcher::_do_forward('Catalyst::Dispatcher=HASH(0x2e052f8)', 'forward', 'MusicBrainz::Server=HASH(0xc203560)', '_ACTION') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Dispatcher.pm line 237
Catalyst::Dispatcher::forward('Catalyst::Dispatcher=HASH(0x2e052f8)', 'MusicBrainz::Server=HASH(0xc203560)', '_ACTION') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 342
Catalyst::forward('MusicBrainz::Server=HASH(0xc203560)', '_ACTION') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Controller.pm line 99
Catalyst::Controller::_DISPATCH('MusicBrainz::Server::Controller::Report=HASH(0xacf5490)', 'MusicBrainz::Server=HASH(0xc203560)', 'ISRCsWithManyRecordings') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Action.pm line 65
Catalyst::Action::execute('Catalyst::Action=HASH(0xb193b88)', 'MusicBrainz::Server::Controller::Report=HASH(0xacf5490)', 'MusicBrainz::Server=HASH(0xc203560)', 'ISRCsWithManyRecordings') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1667
eval {...} called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1667
Catalyst::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb193b88)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Plugin/StackTrace.pm line 94
Catalyst::Plugin::StackTrace::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb193b88)') called at lib/MusicBrainz/Server.pm line 306
MusicBrainz::Server::execute('MusicBrainz::Server=HASH(0xc203560)', 'MusicBrainz::Server::Controller::Report', 'Catalyst::Action=HASH(0xb193b88)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Action.pm line 60
Catalyst::Action::dispatch('Catalyst::Action=HASH(0xb193b88)', 'MusicBrainz::Server=HASH(0xc203560)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Dispatcher.pm line 257
Catalyst::Dispatcher::_do_forward('Catalyst::Dispatcher=HASH(0x2e052f8)', 'forward', 'MusicBrainz::Server=HASH(0xc203560)', '/report/_DISPATCH') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Dispatcher.pm line 237
Catalyst::Dispatcher::forward('Catalyst::Dispatcher=HASH(0x2e052f8)', 'MusicBrainz::Server=HASH(0xc203560)', '/report/_DISPATCH') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 342
Catalyst::forward('MusicBrainz::Server=HASH(0xc203560)', '/report/_DISPATCH') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Dispatcher.pm line 105
Catalyst::Dispatcher::dispatch('Catalyst::Dispatcher=HASH(0x2e052f8)', 'MusicBrainz::Server=HASH(0xc203560)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1613
Catalyst::dispatch('MusicBrainz::Server=HASH(0xc203560)') called at lib/MusicBrainz/Server.pm line 275
Class::MOP::Class:::around('CODE(0x26331b0)', 'MusicBrainz::Server=HASH(0xc203560)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/x86_64-linux-gnu-thread-multi/Class/MOP/Method/Wrapped.pm line 162
Class::MOP::Method::Wrapped::__ANON__('MusicBrainz::Server=HASH(0xc203560)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/x86_64-linux-gnu-thread-multi/Class/MOP/Method/Wrapped.pm line 91
MusicBrainz::Server::dispatch('MusicBrainz::Server=HASH(0xc203560)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1963
Catalyst::__ANON__() called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Try/Tiny.pm line 76
eval {...} called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Try/Tiny.pm line 67
Try::Tiny::try('CODE(0xb30a608)', 'Try::Tiny::Catch=REF(0xc34aa40)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst.pm line 1969
Catalyst::handle_request('MusicBrainz::Server', 'env', 'HASH(0xb2dba80)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Catalyst/Engine.pm line 856
Catalyst::Engine::__ANON__('CODE(0xb30a8c0)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Plack/Handler/FCGI.pm line 134
Plack::Handler::FCGI::run('Plack::Handler::FCGI=HASH(0x9e6d90)', 'CODE(0xb266c58)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Plack/Loader.pm line 88
Plack::Loader::run('Plack::Loader=HASH(0x9af9d0)', 'Plack::Handler::FCGI=HASH(0x9e6d90)') called at /home/musicbrainz/musicbrainz-server/admin/../local/lib/perl5/Plack/Runner.pm line 263
Plack::Runner::run('Plack::Runner=HASH(0x7ab000)') called at /home/musicbrainz/musicbrainz-server/admin/../local/bin/plackup line 10"
Stack trace:
line 224 MusicBrainz::Server::Data::Utils
line 47 MusicBrainz::Server::Report
line 30 MusicBrainz::Server::Report
line 38 MusicBrainz::Server::Controller::Report
line 191 MusicBrainz::Server::Controller
line 40 MusicBrainz::Server::Controller::Report
line 306 MusicBrainz::Server
line 306 MusicBrainz::Server
line 306 MusicBrainz::Server
line 275 MusicBrainz::Server
Request data:
$VAR1 = {
'query_parameters' => {},
'body_parameters' => {}
};
If this is because the reports haven't finished running, I presume it shouldn't be ISEing and should just show the old message about no data being available right now instead. It's because reports don't exist and have never ran. I guess it would be nicer if they didn't crash. This is now live on http://beta.musicbrainz.org/ "Show all results" doesn't work, e.g. The artist column is empty on http://beta.musicbrainz.org/report/ISRCsWithManyRecordings Compare http://musicbrainz.org/report/ReleasesInCAAWithCoverArtRelationships?page=8 http://beta.musicbrainz.org/report/ReleasedTooEarly Artist is also missing from http://beta.musicbrainz.org/report/BadAmazonURLs This has too many open issues and will not make the 2012-07-09 release. It will stay in beta testing and hopefully make the 2012-07-24 release. I believe I've fixed these remaining open isuses and have updated beta. Please let me know if I've missed anything, or you find new problems. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Improvement
Closed
Normal
Allow filtering reports by subscribed entities
. I try to make it ready until end of this week. If this is too late, and you think you can achieve a solution in a much faster time, it is also no problem if you take the ticket after the week...
+1
I just wanted to file the same ticket... the motivation to start cleaning up these huge reports are near zero, it seems like endless work.
Also of course, one probably knows the subscribed entities better than random entities!