Are you really tall???
I'm 1.73cm & I had this issue & my baby was on the 5th, when she was born, yes she's short but was the fattest of all my babies. My babies have got shorter the more I have lol and the introduction of the customised growth charts now is what make them "freak"
The risk of a SGA baby is that if you are going to have a prolonged labour they sometimes don't have the strength to last a prolonged labour & they could loose too much weight in labour.
My first was 2.9 kg, 2nd 3.05 and 3rd was 3.3kg, so when I got the estimated weights I was like this is already bigger than my other babies.
The risk is if your baby isn't growing the required amount between scans, but again they're subjective to who's taking the scan
I was ok cause I was getting 2 weeks std growth between the scans.
OH & the thing I found funny as soon as they're out they're no longer SGA if they're a normal size
They can't make you do anything but can only advise you on what to do. Do you have a midwife or under hospital care? My midwife really went into bat for me & when I got over the hospital invention at about exactly where you are now she stood by my decision. It was the only firm decision I made in my pregnancy but I was backed by a good scan which was still tracking on the 5th. I got so over the hospital I ended up going for a homebirth!!
We also blew off an appointment cause I expected to have her at 39 weeks but instead she hung around til 40.2w.
Was your previous baby overdue?