fix: 403 response from metal archives advanced searched endpoint #2

Closed
opened 2024-04-15 12:16:55 +00:00 by Hazel · 2 comments
Owner

Any search will result in a warning message, that the metal archives endpoint returns 403

2024-04-15 14:15:15 WARNING  WARNING:metal_archives:www.metal-archives.com responded wit 403 at             connection.py:235
                             https://www.metal-archives.com/search/ajax-band-search/?field=name&query=hiii                   
                             &sEcho=1&iColumns=3&sColumns=&iDisplayStart=0&iDisplayLength=200&mDataProp_0=0                  
                             &mDataProp_1=1&mDataProp_2=2. (0-2)    

Most likely it is only the header that need some adjustment.
When fixing this I'd appreciate adding tests, assuming the tests are already in experimental.

Any search will result in a warning message, that the metal archives endpoint returns `403` ``` 2024-04-15 14:15:15 WARNING WARNING:metal_archives:www.metal-archives.com responded wit 403 at connection.py:235 https://www.metal-archives.com/search/ajax-band-search/?field=name&query=hiii &sEcho=1&iColumns=3&sColumns=&iDisplayStart=0&iDisplayLength=200&mDataProp_0=0 &mDataProp_1=1&mDataProp_2=2. (0-2) ``` Most likely it is only the header that need some adjustment. When fixing this I'd appreciate adding tests, assuming the tests are already in experimental.
Hazel added this to the next stable release project 2024-04-15 12:18:08 +00:00
Hazel added the
Kind/Bug
Priority
High
labels 2024-04-15 19:15:18 +00:00
Hazel added reference fix/metal_archives 2024-04-16 11:39:44 +00:00
Author
Owner

Metal archives started to use cloudflare:

image

There is a cloudflare bypass library that should still make it work without it being toooo much work. If this doesn't seem to work, the only option left would be to drop metal archives from the pages, because maintaining a cloudflare bypass would just be too much work.

If metal archives should be dropped, I'd send an email beforehand, asking them to provide an api, or expose theirs.

Metal archives started to use cloudflare: ![image](/attachments/a831773f-af57-480f-85de-eb3ff5d03fed) There is a [cloudflare bypass library](https://github.com/VeNoMouS/cloudscraper) that should still make it work without it being toooo much work. If this doesn't seem to work, the only option left would be to drop metal archives from the pages, because maintaining a cloudflare bypass would just be too much work. If metal archives should be dropped, I'd send an email beforehand, asking them to provide an api, or expose theirs.
Author
Owner

I don't seem to be able to figure out how to bypass cloudfairs. It also seems to be a constant updating task, which probably also requires me to run a browser in the back, and i don't wanna do that.

I don't seem to be able to figure out how to bypass cloudfairs. It also seems to be a [constant updating task](https://stackoverflow.com/questions/73230570/how-to-bypass-cloudflare-with-python), which probably also requires me to run a browser in the back, and i don't wanna do that.
Hazel closed this issue 2024-04-24 08:16:01 +00:00
Sign in to join this conversation.
No Milestone
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: music-kraken/music-kraken-core#2
No description provided.