In their most common words rhyme if they end in identically or nearly identically sounding syllables, and match in stress. If a word has an unusual or unique ending syllable and no other word has a stress pattern to match, it does not rhyme. While many polysyllabic words have no rhyme, one-syllable rhymes are uncommon – maybe fewer than a hundred in the English language and many end in the present or historical suffix -th.
One-syllable words that have no rhymes in the English language include: angst, breadth, bulb, cusp, depth, eighth, fifth, filmed, glimpsed, gulf, ninth, oblige, sculpts, sixth, twelfth, whilst, and the often used example orange. (orange actually rhymes obscurely with Blorenge, a hill in Wales as well as with the word sporange, the singular form of sporangium – the enclosure in which spores are formed.)
Once the stress shifts to the next-to-last syllable, rhyme-less words are quite common, perhaps even the norm. Some examples include: angel, angry, chimney, citrus, elbow, foible, liquid, polka, penguin, vacuum, and many others.