"I'll say sorry, but I'm not taking off my glasses"

The difficulty of n'importe quoi comes not only from it's wildly different translations but also that it makes no sense. With a basic understanding of French, you know that n' vaguely translates as don't, importer means, der, to import, and quoi is what.
Forget all of that. Don't import what! No import what! It will only confuse you.

Soeur: Augustin, arrête de crier !

Frère: Mais les espagnols ! Ils arrivent !

Soeur: N'importe quoi !

=

Sister: Augustin, stop yelling!

Brother: But the Spanish! They're coming!

Sister: Yeah right!

If you want to take it textbook style, this would be translated as nonsense.
It also means whatever, in the "it doesn't matter which" sense.

"Tu veux du gâteau où des crêpes ?"

"Où n'importe quoi."

=

"Do you want cake or some crêpes?"

"Or whatever."

But n'importe can be taken much farther! Here are a few examples of what can be done:

le boucher: Vous voulez lequel ?

Jean Bonnot: N'importe quel !

=

the butcher: Which one do you want?

Jean Bonnot: Whichever!

"Je peux inviter qui ?"

"N'importe qui tu veux ! C'est ton anniversaire !"

=

"Who can I invite?"

"Whoever you want ! It's your birthday!"

"Je fais n'importe quoi je veux, n'importe quand je veux !" = "I do whatever I want, whenever I want!"

N'importe quoi !