I read a lot of writer's and agent's blogs and I have noticed a trend for folks to either cross genres or to claim that their novel lacks a genre. I am aware that some books are difficult to cast into a "Romance" or "Thriller" or "Horror" category and that "Commercial Fiction" or "Literary Fiction" have become sort of the miscellaneous category where books of this ilk are often tossed. But, is saying that your novel doesn't fit a genre meant to elevate it above those categories or merely to differentiate?
I ask this question rhetorically, but the issue with this de-classification or extra-classification is that in the publishing world, often you don't want to be on an island. The sad truth is, if you want readers to potentially happen upon your book on Amazon or in Borders/Barnes & Noble/Books-A-Million, you need to have that book placed in one of their categories.
The next time you are in a bookstore, note the sections. Then watch each section and the traffic. Watch as dozens of teens browse the Manga or Vampire sections, or the fourty-something women pile up Romance books or browse Self-Help, Cookbooks or Religion. If you don't have a "section," your book is hard to "define" and therefore hard to market. Yeah, your coming-of-age novel set in Ireland may be a great read, but unless its on a book list, self promoted like crazy or given a huge marketing budget, fewer people will find it.
Think about how you choose books. Known Author? Cover Art? Browsing on the Internet? Do you have a deliberate way of determining which book will occupy your valuable time and draw from your hard-earned money? If you aren't looking for a specific book, you most often will resort to what I call the "Genre Hunt." You seek out novels in a genre you have predetermined that you will want to read. It is like picking a restaurant. Do you want Mexican, Chinese, or the hard-to-define American food? If you choose the latter, you often are in for a treat--a pie place, a steak place, a burger joint, a greasy spoon, a diner, drive-in or dive. But, isn't it easier to know what you generally want first and then to determine from there what specifically you want?
That is why these genres were created in the first place: to facilitate marketing; to reach the most people in the quickest and simplest way. It has been successful. And if your novel crosses genres? Say you have a Modern Sci-Fi Romance? Fine, you can put it in both genres. You might be surprised that it sells more in Sci-Fi than in the Romance section, but who cares?
Is your novel doomed if it doesn't fall neatly into a category? Why, no. Of course not. It just means you need exceptionally good marketing, which you want anyway. And, of course, you will have to do a fair amount of marketing yourself, which, again, your publisher and agent will probably expect this of you anyway, even if the novel was a pure genre story. The bottom line is, don't define your novel as "undefinable" because you think it elevates your novel above the din of genres. And, don't disdain the categories because they "limit" you and your writing. Embrace them if you can. Use them to show how you differentiate within that genre, not against it. It will allow you to develop an audience and then you can jump out of genres because people will be looking for your NAME. Won't that be nice?