Make links blue or underlined, or both.
Users expect links in web pages to be blue and underlined, because that's the way they appear on 99% of websites.
If you use a different colour then at least the underline is a clue that a link is a link. Likewise, if you remove the underline but keep the link blue, then the colour is the clue that a link is a link. So it's best to use both blue and underlined, but using at least one or the other is acceptable.
What's wrong is doing neither - having links that aren't blue and aren't underlined. We often use different coloured links in our websites if we think the blue colour will distract from the design. But it's most important to ensure the links are obvious and clear.
Links in navigation menus don't have to be blue or underlined, as long as they're clearly menu items. The blue/underlined tip is for links that are in the middle of your web page.
Don't underline words if they're not links.
On the internet, text that's underlined is understood to be a link.
If you underline (and it's not a link), your readers will be annoyed when they try to click those underlined words only to discover that they're not really links.
If you want to emphasize some text, use italics instead (or bold, or another colour).