the Nest will only connect/trigger with other Nest detectors. if you existing hardwire smokes alarm goes off the nest can not silence them. if you just want a way to know remotely if there is a fire or CO issue then the Nest will work for that. if you want to make the house solely Nest then you need one in each bedroom, one outside in a common area of the bedrooms (hallway etc) one in living room/common area, garage, top of stair wells. contact your fire department to see what code would be if you would need additional units. but you are looking at >$500 easy.

another option is to get a Google home/alexa/homepod as they can be set to listen for your current smoke alarms and/or breaking glass and send you an alert to your phone.

Master Electrician. I would highly recommend you either hire or find someone with experience to physically come out and help you. You know enough to be dangerous and far too often I find people giving very dangerous advice. Lutron Cassetta family of smart switches are my go to for any smart lighting. You can find solutions slightly less expensive, but the flexibility and reliable quality I have with Lutron.