The thing about reading Joyce is that he's just smarter than you--his vocabulary is larger, his memory is more prodigious, and he's read far more than you have. That being said, I think the most important thing that Joyce has on the rest of us--and probably the easiest to remedy--is that he just loves language, and is interested in figuring out what he can do with language on the level of the word, the sentence, the paragraph, the chapter, and the book. Everyone else has already covered what's going on at the book/chapter/plot level, so I'll try and shed some light on what's going on at the word/sentence level in terms of innovation. Innovation is the right word to use here, but keep in mind that what's in it for Joyce is ultimately linguistic fun. Think of someone who knows he's good, but doesn't quite know how good and just wants to figure it out. Onward!

First, Playing with Language. My favorite example of this is at the beginning of chapter 7. Joyce has a pair of sentences that go "Grossbooted draymen rolled barrels dullthudding out of Prince's stores and bumped them up on the brewery float. On the brewery float bumped dullthudding barrels rolled by grossbooted draymen out of Prince's stores" (pg 96). The first thing to note is the alliteration--there's a ton of d's and b's clanging around in those sentences, and to a lesser extent the g's, which also ring off the other two letters. In most cases the primary letters are also accompanied by r's. That's just Joyce trying to squeeze in as many similar sounding letters as he can in order to get a kind of amusing bouncy feel. The second thing to note is that the second sentence is basically the first one backwards. That way you get the first sonic symphony all over again, just backwards. Strictly speaking, these sentences add nothing to the plot (or what semblance there is of one) and are kind of just forgotten later, but the point is that it's a sort of amusing use of language, just one fun sounding sentence and then "I wonder if I do this again, but backwards?". These kinds of things are all over the book, and finding the little linguistic quirks is what makes the whole experience so enjoyable.

Next, Plays on Words and Puns. Puns and such are all over this book, and they're something that sticks with Joyce into Finnegan's Wake (which is really just impossible, more on that later). The puns range from the large involved conceptual puns--like the character in the cyclops chapter that is always referred to as "says I," with "I" as in "eye" as in cyclopses are known for having only one eye--to the more vulgar play on words "poronosophical." You could write an academic treatise--and I'm sure someone has--on how all the various puns in Ulysses link ideas together or subvert the pseudolinguistical such-and-such, but I prefer to just enjoy them for the witty plays on words that they are. Another good example is in Circe (ch 15) when Bloom imagines he's become a priest and he spouts of "Aleph Beth Ghimel Daleth Hagadah Tephilim Kosher Yom Kippur Hanukah Roschaschana Beni Brith Bar Mitzvah.." (pg 397) you get the picture, a lot of stereotypically Jewish words in a nonsensical sequence, then in the next line someone refers to him as "His Most Catholic Majesty." It's not laugh out loud funny, and it sped right by me the first time I read it, but now it kind of makes me chuckle a bit at best, and at worst I'm bemused with irony. These little ironies and jokes are all over the book, you just have to pay an absurd amount of attention to find them.

Last, the Meta-Stuff. I said Joyce had a more prodigious memory than you and I--here's proof. Ulysses is exceedingly self-referential. Here are a couple of the easiest places to see it. At the beginning of Ch 11 (pg 210, don't remember what its name is) there's a longish section spanning a little under two pages that reads as one of those frustratingly nonsensical (post)modernist poems, where one phrase has nothing to do with the next. Most of those phrases (I want to say all, but I've never gone and checked each one) pop up in one form or another throughout the rest of the chapter once the beginning bit ends with "Begin!" (another little joke). You can see it the easiest in the first words of the chapter which are "Bronze by gold,"--also the very first words after "Begin!" Chapter 15, as well, is full of these types of self-referential lines. One particularly good spot is when "The Daughters of Erin" say "Kidney of Bloom, pray for us / Flower of Bath, pray for us / ... Wandering Soap, pray for us / Sweets of Sin, pray for us ... / Potato Preservative against Plague and Pestilence, pray for us" (pg 407) All of the things in the list are things that occur in the book, mostly things that Bloom does or encounters or possesses throughout the book. The kidney is the same as the kidney that Bloom eats when we first see him. Sweets of Sin is the book he picks up (I think, that or it's the one his Wife is reading, it's been a while since I've read Ulysses). The wandering soap refers to the bit of soap that floats down the river throughout the entire book (you can track this soap throughout many of the chapters in the book--look for it, it's kind of fun to see if you can find where the floating soap pops up). My favorite one is the Potato--Bloom carries a shriveled potato in his pocket as a good luck charm (he's an Irishman, har har). The other thing Joyce does is allude to things. well, to everything. I don't have examples of these off the top of my head, but the bible is a big thing he references (particularly in passages that deal with Stephen), Shakespeare, as well--Greek/Roman literature, obviously, hence Ulysses.

The last thing I have to say is that if you're really interested in learning about Joyce, read him in order. Start with the Dubliners--which is one of the best short story collections in all of English literature. It gives you a sense of where Joyce started, and one of the themes that really interests him--people, social interactions. The Dubliners is a set of stories about people living in, you guessed it, Dublin. The stories interact with each other enough to make it a stellar collection, but not enough that you could term it a novel. Then move on to Portrait of the Artist--that'll give you all the Stephen Dedalus you could possibly want, and it starts to show Joyce's stylistic shift to something more "experimental" and more radically psychologically "internal." Then there's Ulysses, which moves full force into the experimental and the psychologically internal, but it does sort of what Dubliners does, it creates a portrait of the city, except this time we have a main character, and the interactions between the different stories going on becomes more apparent. Then Finnegan's Wake. I don't recommend this, as it's not actually enjoyable to read because it's so dense and obscure. The language in it is lovely, the jokes are hilarious (when you can get them) and Joyce is at his most innovative, but on the whole, it's entirely too obscure. If you thought the "says I" to cyclops pun was a stretch, then Finnegan's Wake is certainly not for you. There are tons of multilingual puns, yes, that's right, Joyce is punning between different languages, and the sound play is surpassed only by Hopkins, but plot... well, there really isn't one--sort of, we think.. scholars are actually in debate about this. Same with characters, they come and go, there's only "Here Comes Everybody" the main character and his relations, who I find difficult to keep track of.

Last thing: if you want a good guide to the book, i suggest this one as it has a different Joyce Scholar introduce and walk you through each chapter, it's the one I ended up using when I went through Ulysses the first time.

TL;DR Pay attention to the individual bits of language that Joyce is using, there are plenty of little jokes and ironies there. Also, fuck Finnegan's Wake.

Also also, page numbers refer to the "Garbler Edition" of Ulysses.