Federal tax status and state corporate status are two entirely separate things.  If you are going out of business (or if you are out of business already), you really ought to take care of both situations.

It is simple to file a “final” federal tax return and let the IRS know that you are going out of business.  That will clear your status with the IRS.  If you just allow the IRS to revoke your exemption, it may come after you for for-profit corporate tax returns and could impose penalties for failure to file them.  As long as the corporation continues to exist, it generally has an obligation to file even if it has no income or expense.

You can avoid that obligation by ceasing to exist legally, i.e. by dissolving at the state level.  If you don’t dissolve at the state level, the corporation will continue to exist, your board will continue to have fiduciary duties, and the state may look for required state reports.  Everyone will rest more easily if you do it right and dissolve.