It is really hard to be around the food you love that you can no longer eat. I was recently diagnosed as celiacs and I hate not being able to have croissants, it is hard when my friends suggest going to the bakery/ coffee shop for lunch, sometimes we find somewhere else and sometimes we go where they want. But my allergy is not life threatening, if it were just that however I would say you are being a Jerk. But since you have an allergy that seems to be getting worse with increased exposure it would seem like the next time could be life threatening, therefore asking your roommates to not bring food that can kill you if you can smell it is reasonable. I have had coworkers and fellow students who were allergic to strawberry and mango and have to request that the people that are commonly around them not bring those foods to school nor to wear body products that use components. If it is normal to do that in a classroom surely it is normal to do in in your own home. being able to breathe is super important. NTA.