So I think my heater core is out on my Cavalier, would it be ok to just disconnect the coolant hoses going into the firewall and connecting them together to bypass the core for now? Would this cause the engine to overheat at all?

My car started dumping coolant out of the L shaped elbow coming out of the firewall, and when I checked by turning my heater on, the air smelled like coolant pretty bad. I let the car cool down for half an hour, and tried it again, but this time it wasn't blowing hot air.

Yes you can temporarily bypass the heater core and cause no harm.Just make sure when you replace the heater core buy new heater core hoses and clamps.Also may be a good time to upgrade the thermostat and upper/lower radiator hoses all at once.If these hoses are over 4yrs old it is recommended to replace them especially in extreme hot or cold due to failure because of temps.I do my hoses every four years a low cost piece of mind when flushing the cooling system.