fix: 403 response from metal archives advanced searched endpoint #2

New Issue

Hazel · 2024-04-15T12:16:55Z

Hazel commented

2024-04-15 12:16:55 +00:00

Any search will result in a warning message, that the metal archives endpoint returns 403

2024-04-15 14:15:15 WARNING  WARNING:metal_archives:www.metal-archives.com responded wit 403 at             connection.py:235
                             https://www.metal-archives.com/search/ajax-band-search/?field=name&query=hiii                   
                             &sEcho=1&iColumns=3&sColumns=&iDisplayStart=0&iDisplayLength=200&mDataProp_0=0                  
                             &mDataProp_1=1&mDataProp_2=2. (0-2)

Most likely it is only the header that need some adjustment.
When fixing this I'd appreciate adding tests, assuming the tests are already in experimental.

Any search will result in a warning message, that the metal archives endpoint returns `403` ``` 2024-04-15 14:15:15 WARNING WARNING:metal_archives:www.metal-archives.com responded wit 403 at connection.py:235 https://www.metal-archives.com/search/ajax-band-search/?field=name&query=hiii &sEcho=1&iColumns=3&sColumns=&iDisplayStart=0&iDisplayLength=200&mDataProp_0=0 &mDataProp_1=1&mDataProp_2=2. (0-2) ``` Most likely it is only the header that need some adjustment. When fixing this I'd appreciate adding tests, assuming the tests are already in experimental.

Hazel added this to the next stable release project 2024-04-15 12:18:08 +00:00

Hazel added the

Kind/Bug

Priority

High

labels 2024-04-15 19:15:18 +00:00

Hazel added reference fix/metal_archives

2024-04-16 11:39:44 +00:00

Hazel commented

2024-04-16 12:10:03 +00:00

Metal archives started to use cloudflare:

There is a cloudflare bypass library that should still make it work without it being toooo much work. If this doesn't seem to work, the only option left would be to drop metal archives from the pages, because maintaining a cloudflare bypass would just be too much work.

If metal archives should be dropped, I'd send an email beforehand, asking them to provide an api, or expose theirs.

Metal archives started to use cloudflare: ![image](/attachments/a831773f-af57-480f-85de-eb3ff5d03fed) There is a [cloudflare bypass library](https://github.com/VeNoMouS/cloudscraper) that should still make it work without it being toooo much work. If this doesn't seem to work, the only option left would be to drop metal archives from the pages, because maintaining a cloudflare bypass would just be too much work. If metal archives should be dropped, I'd send an email beforehand, asking them to provide an api, or expose theirs.

image.png

54 KiB

Hazel referenced this issue

2024-04-23 10:03:58 +00:00

fix/bandcamp #12

Hazel commented

2024-04-24 08:16:01 +00:00

I don't seem to be able to figure out how to bypass cloudfairs. It also seems to be a constant updating task, which probably also requires me to run a browser in the back, and i don't wanna do that.

I don't seem to be able to figure out how to bypass cloudfairs. It also seems to be a [constant updating task](https://stackoverflow.com/questions/73230570/how-to-bypass-cloudflare-with-python), which probably also requires me to run a browser in the back, and i don't wanna do that.

Hazel closed this issue

2024-04-24 08:16:01 +00:00

Sign in to join this conversation.