Remove memory optimized filegroup

I read this:http://www.sqlservercentral.com/Forums/Topic1568262-32-1.aspxWhich says you must drop the database to remove the filegroup.I deleted all the objects and then backed up the DB and restored it and the filegroup is still there. I was skeptical but some of the comments made me think this might work.Do I really have to restore from a pre-memory optimized state?Thank you,Josh