Because Google doesn’t verify that the number you enter belongs to you, it is easy to use the service for pranks and to harass people.
A relatively benign use would be to have the service call people in the middle of the night and have the business blamed for waking up someone. A more serious scenario would be to have the service connect someone to an escort service. (“Honey, why is there a call from an escort service on the caller ID?”)
This type of use is less dangerous than some of the caller ID spoofs out there because it connects the recipient to a known number. For example, having someone connected to Bank of America will likely just cause confusion and can’t be used for phishing.
But there is enough potential for causing trouble that it’s worth exploring mechanisms to address it.
One solution is to only display the business’ number on the caller ID when the user is logged in and has verified that the number is associated with their account. For some people, this will raise privacy issues with Google having phone numbers associated with their account.
Calls to numbers not associated with the account (or from unauthenticated users) would have a Google number displayed on the caller ID, with a preamble that says something along the lines of “This is Google’s click to call service. Please hold to be connected to the business. If you did not initiate this call press 1 for more information.”