You probably want the %{REMOTE_ADDR} to match against, but you can't use CIDR notation as the %{REMOTE_ADDR} is literally the remote address and you can use a regular expression to try to match against it. So for 123.45.67.89/28, (123.45.67.80 - 123.45.67.95), you'd have to do something like this:
RewriteCond %{REMOTE_ADDR} !^123\.45\.67\.8[0-9]$
RewriteCond %{REMOTE_ADDR} !^123\.45\.67\.9[0-5]$