There are two ways to turn off active alerts...
1) You can disable an alert: this preserves the alert settings and halts the alert notifications until it is re-enabled.
2) You can delete an alert: this completely removes the configured alert.
From the main menu, go to Alerts > Edit My Alerts > choose an alert.
To disable: toggle Enabled to the "off" position (left-hand screenshot)
To delete: tap Delete Alert at the very bottom (right-hand screenshot)
(Note: In the mobile applications, only push-notification alerts will be visible. You must visit the website to edit all other alerts.)
From the home page, select My Alerts from the My Weather drop down menu.
Then choose to disable (pause icon) or delete ("x" icon) the alert.
Alternatively, you can disable the alert method by un-checking the configured devices.