feature/artwork_gallery #41

Merged

Hazel merged 25 commits from feature/artwork_gallery into experimental 2024-07-15 09:36:22 +00:00

Conversation 34 Commits 25 Files Changed 20 +774 -332

Luna commented 2024-06-04 09:13:13 +00:00

Owner

Copy Link

No description provided.

Luna added 7 commits 2024-06-04 09:13:13 +00:00

Artwork gallery Musify 5d26fdbf94

feat: added parent artwork options bc19a94e7f

feat: added hooks for collection on append 49c3734526

feat: added artist.artwork to data structure eb8fd5e580

feat: added album.artwork to datastructure 1ef4b27f28

feat: musify completed 05ee09e25f

feat: structure changes to artwork and collection objects d51e3a56fb

Hazel commented 2024-06-04 09:15:06 +00:00

Owner

Copy Link

Please add the following settings:

image_format = "jpg"
download_artist_artworks = true
artist_artwork_path = "{genre}/{artist}/{artist}.{image_format}"

Please add the following settings: ``` image_format = "jpg" download_artist_artworks = true artist_artwork_path = "{genre}/{artist}/{artist}.{image_format}" ```

Luna added 1 commit 2024-06-04 09:44:56 +00:00

feat: config changes d83e40ed83

Luna added 1 commit 2024-06-05 06:34:46 +00:00

feat: bandcamp artist artwork 7d23ecac06

Luna added 1 commit 2024-06-05 07:47:10 +00:00

feat: fix saving img in tmp 3118140f0f

Hazel added 1 commit 2024-06-05 10:05:46 +00:00

feat: renamed artwork df98a70717

Luna added 2 commits 2024-06-05 11:49:57 +00:00

feat: youtube music album and artist artwork d447b10380

Merge branch 'feature/artwork_gallery' of ssh://gitea.elara.ws:2222/music-kraken/music-kraken-core into feature/artwork_gallery 8e3ec0f4ed

Hazel added 2 commits 2024-06-06 15:53:52 +00:00

draft implemented add_data 4e50bb1fba

Merge branch 'feature/artwork_gallery' of ssh://gitea.elara.ws:2222/music-kraken/music-kraken-core into feature/artwork_gallery 01dffc2443

Luna commented 2024-06-06 15:58:35 +00:00

Author

Owner

Copy Link

Is the Interface ready?

Is the Interface ready?

Hazel added 1 commit 2024-06-07 09:15:30 +00:00

feat: removed distracting code eef3ea7f07

Hazel added 1 commit 2024-06-07 09:17:53 +00:00

feat: added extend 346d273201

Hazel commented 2024-06-07 09:23:38 +00:00

Owner

Copy Link

Is the Interface ready?

@Luna not yet, but the code is pushed. I suggest you look at it and try implementing it yourself. Here is how the interface should work:

Classes

• ArtworkCollection: All the different artworks for one data objects
• Artwork: One artwork with multiple ArtworkVariants. This is necessary because maybe the same picture can be found with multiple different urls
• ArtworkVariant: Contains the actual url of the artwork

Functions

The most important function for the scraping is ArtworkCollection.add_data, which adds the data passed into the function to existing entities or new one. It matches new ones by the url. Artwork.add_data does the same. This should be implemented.

For consistency ArtworkCollection.append has to exist even though it probaply won't be used much. There are 2 options how to go about it.

• You can pass in a dict of data, an artworkvariant or an artwork
• You can only pass in an artwork
  The second option would be cleaner imo but I want your opppinion.

ArtworkCollection.compile fetches every variant of artworks, and is called once the data-object is compiled (aka on download). There should the duplicate recognition be implemented

---

unfortunately I couldn't run the changes because of time so you will probaply will have to fix many small issues because of so big changes.

> Is the Interface ready? @Luna not yet, but the code is pushed. I suggest you look at it and try implementing it yourself. Here is how the interface should work: # Classes - `ArtworkCollection`: All the different artworks for one data objects - `Artwork`: One artwork with multiple `ArtworkVariants`. This is necessary because maybe the same picture can be found with multiple different urls - `ArtworkVariant`: Contains the actual url of the artwork # Functions The most important function for the scraping is `ArtworkCollection.add_data`, which adds the data passed into the function to existing entities or new one. It matches new ones by the url. `Artwork.add_data` does the same. This should be implemented. For consistency `ArtworkCollection.append` has to exist even though it probaply won't be used much. There are 2 options how to go about it. - You can pass in a dict of data, an artworkvariant or an artwork - You can only pass in an artwork The second option would be cleaner imo but I want your opppinion. `ArtworkCollection.compile` fetches every variant of artworks, and is called once the data-object is compiled (aka on download). There should the duplicate recognition be implemented --- unfortunately I couldn't run the changes because of time so you will probaply will have to fix many small issues because of so big changes.

Hazel added 1 commit 2024-06-07 09:28:01 +00:00

feat: added compile 2da7a48b72

Luna commented 2024-06-07 09:36:22 +00:00

Author

Owner

Copy Link

Perfect thanks I'll probably do that tonight

Perfect thanks I'll probably do that tonight

Hazel commented 2024-06-07 09:40:15 +00:00

Owner

Copy Link

Duplicate Recognition of the images

This comment is only about removing duplicates

@Elara6331 said Google uses the simhash algorythm for their web crawler, and it doesn't seem to bad to implement.

@Luna found this code in a cli tool, which looks really nice:

from PIL import Image
import imagehash

for img in sorted(image_filenames):
    try:
        hash = imagehash.average_hash(Image.open(img))
    except Exception as e:
        print('Problem:', e, 'with', img)
        continue

    if hash in images:
        print(img, ' already exists as', ' '.join(images[hash]))
    if 'dupPictures' in img:
        print('rm -v', img)
        images[hash] = images.get(hash, []) + [img]

It uses ImageHash, which looks REALLY good and has 3k Stars, so this seems to be the best option.

One thing to note is, that this can calculate the difference between the images, and I would just add it to the settings at which threshhold the images are duplicates or not.

>>> from PIL import Image
>>> import imagehash
>>> hash = imagehash.average_hash(Image.open('tests/data/imagehash.png'))
>>> print(hash)
ffd7918181c9ffff
>>> otherhash = imagehash.average_hash(Image.open('tests/data/peppers.png'))
>>> print(otherhash)
9f172786e71f1e00
>>> print(hash == otherhash)
False
>>> print(hash - otherhash)  # hamming distance
33

# Duplicate Recognition of the images > This comment is only about removing duplicates @Elara6331 said Google uses the simhash algorythm for their web crawler, and it doesn't seem to bad to implement. @Luna found this code in a cli tool, which looks really nice: ```python from PIL import Image import imagehash for img in sorted(image_filenames): try: hash = imagehash.average_hash(Image.open(img)) except Exception as e: print('Problem:', e, 'with', img) continue if hash in images: print(img, ' already exists as', ' '.join(images[hash])) if 'dupPictures' in img: print('rm -v', img) images[hash] = images.get(hash, []) + [img] ``` It uses [ImageHash](https://pypi.org/project/ImageHash/), which looks REALLY good and has 3k Stars, so this seems to be the best option. One thing to note is, that this can calculate the difference between the images, and I would just add it to the settings at which threshhold the images are duplicates or not. ```python >>> from PIL import Image >>> import imagehash >>> hash = imagehash.average_hash(Image.open('tests/data/imagehash.png')) >>> print(hash) ffd7918181c9ffff >>> otherhash = imagehash.average_hash(Image.open('tests/data/peppers.png')) >>> print(otherhash) 9f172786e71f1e00 >>> print(hash == otherhash) False >>> print(hash - otherhash) # hamming distance 33 ```

Luna commented 2024-06-07 09:51:49 +00:00

Author

Owner

Copy Link

👍 so no change for image hashing should be the same libary that you found

👍 so no change for image hashing should be the same libary that you found

Hazel commented 2024-06-07 09:57:47 +00:00

Owner

Copy Link

👍 so no change for image hashing should be the same libary that you found

If @Elara6331 or you don't have good arguments that overweigh the pros of this library then yes 👍

But I highly doubt this is controversioal at all

> 👍 so no change for image hashing should be the same libary that you found If @Elara6331 or you don't have good arguments that overweigh the pros of this library then yes 👍 But I highly doubt this is controversioal at all

Elara6331 commented 2024-06-07 14:25:12 +00:00

Owner

Copy Link

If @Elara6331 or you don't have good arguments that overweigh the pros of this library then yes 👍

ImageHash looks good to me :3

> If @Elara6331 or you don't have good arguments that overweigh the pros of this library then yes 👍 ImageHash looks good to me :3

Luna commented 2024-06-07 16:39:03 +00:00

Author

Owner

Copy Link

ArtworkCollection.compile fetches every variant of artworks, and is called once the data-object is compiled (aka on download). There should the duplicate recognition be implemented

there is a slight problem with that, since artwork is an object i cant import download since download itself imports artwork so question would be what i should do with that problem.

> `ArtworkCollection.compile` fetches every variant of artworks, and is called once the data-object is compiled (aka on download). There should the duplicate recognition be implemented > there is a slight problem with that, since artwork is an object i cant import download since download itself imports artwork so question would be what i should do with that problem.

Luna commented 2024-06-07 16:43:26 +00:00

Author

Owner

Copy Link

there is a slight problem with that, since artwork is an object i cant import download since download itself imports artwork so question would be what i should do with that problem.

wouldnt even be that bad just means that i cant check for duplicates in artwork

> there is a slight problem with that, since artwork is an object i cant import download since download itself imports artwork so question would be what i should do with that problem. wouldnt even be that bad just means that i cant check for duplicates in artwork

Hazel commented 2024-06-07 19:24:23 +00:00

Owner

Copy Link

Yes I am aware of this issue. Fuck circular imports, so easy to avoid when designing a language. Because of technicalities it works if you import the connection class in the Artwork constructor. Then just create an instance of the class in the constructor right after. Just give it a constant module name. Cuz then I can optimize the collection class to only create one global instance of each module later <333

Yes I am aware of this issue. Fuck circular imports, so easy to avoid when designing a language. Because of technicalities it works if you import the connection class in the Artwork constructor. Then just create an instance of the class in the constructor right after. Just give it a constant module name. Cuz then I can optimize the collection class to only create one global instance of each module later <333

Hazel commented 2024-06-07 19:24:23 +00:00

Owner

Copy Link

Yes I am aware of this issue. Fuck circular imports, so easy to avoid when designing a language. Because of technicalities it works if you import the connection class in the Artwork constructor. Then just create an instance of the class in the constructor right after. Just give it a constant module name. Cuz then I can optimize the collection class to only create one global instance of each module later <333

Yes I am aware of this issue. Fuck circular imports, so easy to avoid when designing a language. Because of technicalities it works if you import the connection class in the Artwork constructor. Then just create an instance of the class in the constructor right after. Just give it a constant module name. Cuz then I can optimize the collection class to only create one global instance of each module later <333

Luna commented 2024-06-07 19:35:26 +00:00

Author

Owner

Copy Link

Okay thanks ugly way to do it (not your fault) but it's a way xD thanks for the help ❤️🫂

Okay thanks ugly way to do it (not your fault) but it's a way xD thanks for the help ❤️🫂

Luna commented 2024-06-07 19:35:27 +00:00

Author

Owner

Copy Link

Okay thanks ugly way to do it (not your fault) but it's a way xD thanks for the help ❤️🫂

Okay thanks ugly way to do it (not your fault) but it's a way xD thanks for the help ❤️🫂

❤️ 1

Hazel commented 2024-06-07 19:41:30 +00:00

Owner

Copy Link

@Elara6331 you gonna love this fix and limitation xD

@Elara6331 you gonna love this fix and limitation xD

Luna commented 2024-06-07 19:45:48 +00:00

Author

Owner

Copy Link

With that issue away I think I'll have it done tonight excepte the metadata stuff not really sure where to put what 🙃

With that issue away I think I'll have it done tonight excepte the metadata stuff not really sure where to put what 🙃

Elara6331 commented 2024-06-07 21:33:04 +00:00

Owner

Copy Link

@Elara6331 you gonna love this fix and limitation xD

Python allows circular imports?!
Also, how does that workaround work?? This makes no sense. Circular imports should just be disallowed completely.

The more I learn about Python, the more convinced I am that it was designed by a drunk 5 year old with no programming experience lol.

> @Elara6331 you gonna love this fix and limitation xD Python allows circular imports?! \ Also, how does that workaround work?? This makes no sense. Circular imports should just be disallowed completely. The more I learn about Python, the more convinced I am that it was designed by a drunk 5 year old with no programming experience lol.

Luna commented 2024-06-07 21:46:29 +00:00

Author

Owner

Copy Link

Still better than java c# and many other languages

Still better than java c# and many other languages

Luna commented 2024-06-07 21:51:27 +00:00

Author

Owner

Copy Link

I hope you meant circular imports shouldn't exist

I hope you meant circular imports shouldn't exist

Luna commented 2024-06-07 21:52:05 +00:00

Author

Owner

Copy Link

Because it's really really pointless and annoying

Because it's really really pointless and annoying

Elara6331 commented 2024-06-07 21:52:14 +00:00

Owner

Copy Link

I hope you meant circular imports shouldn't exist

Yes, that's what I meant. Circular imports should not be a thing.

> I hope you meant circular imports shouldn't exist Yes, that's what I meant. Circular imports should not be a thing.

Luna commented 2024-06-07 21:58:15 +00:00

Author

Owner

Copy Link

I think not even java has that "feature" and that says a lot 😂

I think not even java has that "feature" and that says a lot 😂

Luna commented 2024-06-08 10:27:03 +00:00

Author

Owner

Copy Link

How can it be so hard to import with that annoying import function

How can it be so hard to import with that annoying __import__ function

Luna commented 2024-06-08 20:46:20 +00:00

Author

Owner

Copy Link

I kinda dont know how to fix this i cant really import the connection module because it imports some other modules and it just doesnt work and i amout of ideas

I kinda dont know how to fix this i cant really import the connection module because it imports some other modules and it just doesnt work and i amout of ideas

Luna commented 2024-06-09 08:39:55 +00:00

Author

Owner

Copy Link

Since i am kinda to dumb to handle this i am gona put the compile function into page_attributes if any of you have a better idea just tell me.

Since i am kinda to dumb to handle this i am gona put the compile function into page_attributes if any of you have a better idea just tell me.

Hazel commented 2024-06-10 10:01:08 +00:00

Owner

Copy Link

@Luna I can't look at it, if you don't push your changes.

@Luna I can't look at it, if you don't push your changes.

Luna commented 2024-06-10 10:19:55 +00:00

Author

Owner

Copy Link

Sorry

Sorry

Luna added 1 commit 2024-06-10 13:03:13 +00:00

feat:a lot of nonsences 4ee6fd2137

Hazel added 2 commits 2024-06-11 12:54:47 +00:00

fix: implemented artwork.add_data b1a306f3f3

feat: implemented fetching of artworks on compile 274f1bce90

Hazel added 1 commit 2024-06-11 12:58:12 +00:00

fix: circular input dd99e60afd

Hazel commented 2024-06-11 12:59:14 +00:00

Owner

Copy Link

@Luna I fixed the add_data function, and I fixed the circular import in a very clean way. Now you should be able to continue <333

@Luna I fixed the `add_data` function, and I fixed the circular import in a very clean way. Now you should be able to continue <333

Luna commented 2024-06-11 13:00:49 +00:00

Author

Owner

Copy Link

Your the bestttt 🥹❤️❤️❤️❤️ thank you so so much

Your the bestttt 🥹❤️❤️❤️❤️ thank you so so much

Luna commented 2024-06-11 13:11:27 +00:00

Author

Owner

Copy Link

Wait wait wait you fixed the circulat import issue with just one line of code ?!!!???
Also does this mean i can do the download in the artwork class?

Wait wait wait you fixed the circulat import issue with just one line of code ?!!!??? Also does this mean i can do the download in the artwork class?

Hazel commented 2024-06-11 15:49:17 +00:00

Owner

Copy Link

Wait wait wait you fixed the circulat import issue with just one line of code ?!!!???
Also does this mean i can do the download in the artwork class?

Yes. All I did was only import the objects module for the type checker. Thus you still have the intellisense, but no circular import :)

> Wait wait wait you fixed the circulat import issue with just one line of code ?!!!??? > Also does this mean i can do the download in the artwork class? > > Yes. All I did was only import the objects module for the type checker. Thus you still have the intellisense, but no circular import :)

Luna commented 2024-06-11 15:57:42 +00:00

Author

Owner

Copy Link

That's so crazy your the best ❤️

That's so crazy your the best ❤️

❤️ 1

Luna added 1 commit 2024-06-17 12:50:47 +00:00

feat: musify ArtworkCollection simple function 17c28722fb

Luna added 1 commit 2024-07-01 13:00:01 +00:00

feat: image hash implemented 93c9a367a2

Luna commented 2024-07-02 08:38:30 +00:00

Author

Owner

Copy Link

Musify is done genius and bandcamp should also work but something is majorly wrong with Youtube music invideous isn't even responding i am gona investigate further and write everything that i found but i dont think i am qualified to fix this issue.

Musify is done genius and bandcamp should also work but something is majorly wrong with Youtube music invideous isn't even responding i am gona investigate further and write everything that i found but i dont think i am qualified to fix this issue.

Luna added 1 commit 2024-07-02 15:20:36 +00:00

feat: genius fixes and duplicate detection 5ce76c758e

Luna commented 2024-07-02 15:23:13 +00:00

Author

Owner

Copy Link

Okay i am done it would be great if @Hazel could look over it and remove the wip tag if your happy with the result.

Okay i am done it would be great if @Hazel could look over it and remove the wip tag if your happy with the result.

Luna changed title from WIP: feature/artwork_gallery to feature/artwork_gallery 2024-07-03 11:55:06 +00:00

Hazel merged commit 810aff4163 into experimental 2024-07-15 09:36:22 +00:00

Hazel referenced this issue from a commit 2024-07-15 09:36:22 +00:00

Merge pull request 'feature/artwork_gallery' (#41) from feature/artwork_gallery into experimental

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

Kind/Breaking

Breaking change that won't be backward compatible

  

Kind/Bug

Something is not working

  

Kind/Documentation

Documentation changes

  

Kind/Enhancement

Improve existing functionality

  

Kind/Feature

New functionality

  

Kind/Security

This is security issue

  

Kind/Testing

Issue or pull request related to testing

  

Priority

Critical

The priority is critical

  

Priority

High

The priority is high

  

Priority

Low

The priority is low

  

Priority

Medium

The priority is medium

  

Reviewed

Confirmed

Issue has been confirmed

  

Reviewed

Duplicate

This issue or pull request already exists

  

Reviewed

Invalid

Invalid issue

  

Reviewed

Won't Fix

This issue won't be fixed

  

Status

Abandoned

Somebody has started to work on this but abandoned work

  

Status

Blocked

Something is blocking this issue or pull request

  

Status

Need More Info

Feedback is required to reproduce issue or to continue work

No Label

Kind/Breaking

Kind/Bug

Kind/Documentation

Kind/Enhancement

Kind/Feature

Kind/Security

Kind/Testing

Priority

Critical

Priority

High

Priority

Low

Priority

Medium

Reviewed

Confirmed

Reviewed

Duplicate

Reviewed

Invalid

Reviewed

Won't Fix

Status

Abandoned

Status

Blocked

Status

Need More Info

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

3 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: music-kraken/music-kraken-core#41

No description provided.