One I Actually Never Heard Before

Roy offers:

There was a young curate of Kew
Who kept a tom cat in a pew;
He taught it to speak
Alphabetical Greek
But it never got farther than μ.

I first read it as "never got farther than p" which also works, I think. And I do recall:

There was a young curate of Kew
Whose limericks stopped at line two.

And, of course:

There was a young man from Verdun.  

And needless to say, it's fine to learn a limerick that starts with "a young curate of Kew," and does not end by grossing you out.