from broad category to specifics within category: genre/topic -> alphabetically by author -> alphabetically by book title or by series number color may look nice but how tf can you find anything if you don't remember the color of the front cover? seems like it will only be useful in rare instances but not as the basis of an organizing system
Donโt have OCD so ig it depends specifically on how your symptoms manifest and what your intrusive thoughts are (OCD doesnโt sound fun tho so I wish you the best!!) but I put way too much stock in how my bookshelves are organized (idk if I could function if someone messed them up) and I have super strong opinions
I used to separate by genre and alphabetize but it was so much work and I had to move everything every time I got a new book. Now I sort by having all my favorites in the middle shelf and branch out from there, seriesโ are together and it allows for nicer displays as I can just pick what fits/looks nice over what is the next in order
So for example, I have my nonfiction shelf. One shelf of biographies organized by subject locality (ie American, British, German, etc) and subcategorized by time period. A second shelf of memoirs with the same categorization. One shelf of history- American history by time period and then non-American history (I unfortunately donโt have a large selection of these). And then one science shelf that I have a more or less continuous gradient from genetics/biology to anthropology/sociology to physics
Fiction shelves I have a mystery/thrillers shelf organized from fluffy to gritty/dark and then a shelf for John Grishom. I have a Stephen King shelf organized by subgenre, typically fluffy to gritty, and then the few other horror-ish novels I have. My fantasy/sci-fi shelf is my least organized, I just go by my personal feeling about the author. And then general fiction I sort both by feelings/excitement about the book and diversity of the author/contents
Alphabetical by authorโs last name is acceptable imo, and/or size if space is an issue, but alpha by book title is atrocious and chaotic (splitting series and authors is anathema) and by color looks pretty but is organizationally useless imo (plus youโll still be splitting authors and series)