Replace number in string with ? and /;

I am trying to make the following:

StringReplace[{"A4", "A#3"}, {"A" ~~ x_ -> "B" ~~ x}]

This should replace A4 by B4 but leave A#3 untouched. How can I do that? I tried to use ? and /; with a test for whether or not the _ is a number string as follows:

StringReplace[{"A4","A#3"}, {"A" ~~ x_?StringMatchQ[x_, NumberString] -> "B" ~~ x}]

But it doesn't work. I don't know how to make the syntax work for this.