Technical Thursday – Bug in azure-cli 2.0.30
In this week I would like to inform you about a bug in azure-cli 2.0.30 which can cause some inconveniences when you want to copy blobs in Azure storage accounts.
Some days ago I started to create a solution for copy files inside storage account (this is related a git pipeline solution) and I was facing an issue when I wanted to use ” az storage blob copy start” command with “–sas-token” parameter. The command was quite simple:
az storage blob copy start --account-name mystorageacc --sas-token "?sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************" --destination-container "files" --source-container "files" --source-blob "new/arm-template.json" --destination-blob "archive/arm-template.json"
and I received the following error:
The specified resource does not exist.ErrorCode: CannotVerifyCopySource
<?xml version="1.0" encoding="utf-8"?><Error><Code>CannotVerifyCopySource</Code><Message>The specified resource does not exist.
RequestId:fe725383-701e-0002-5aae-ccdd02000000
Time:2018-04-05T07:20:26.8836489Z</Message></Error>
Traceback (most recent call last):
File "/usr/lib64/az/lib/python2.7/site-packages/knack/cli.py", line 197, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 347, in execute
six.reraise(*sys.exc_info())
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 319, in execute
result = cmd(params)
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/commands/__init__.py", line 180, in __call__
return super(AzCliCommand, self).__call__(*args, **kwargs)
File "/usr/lib64/az/lib/python2.7/site-packages/knack/commands.py", line 109, in __call__
return self.handler(*args, **kwargs)
File "/usr/lib64/az/lib/python2.7/site-packages/azure/cli/core/__init__.py", line 420, in default_command_handler
result = op(**command_args)
File "/usr/lib64/az/lib/python2.7/site-packages/azure/multiapi/storage/v2017_07_29/blob/baseblobservice.py", line 3032, in copy_blob
False)
File "/usr/lib64/az/lib/python2.7/site-packages/azure/multiapi/storage/v2017_07_29/blob/baseblobservice.py", line 3102, in _copy_blob
return self._perform_request(request, _parse_properties, [BlobProperties]).copy
File "/usr/lib64/az/lib/python2.7/site-packages/azure/multiapi/storage/v2017_07_29/common/storageclient.py", line 354, in _perform_request
raise ex
AzureMissingResourceHttpError: The specified resource does not exist.ErrorCode: CannotVerifyCopySource
<?xml version="1.0" encoding="utf-8"?><Error><Code>CannotVerifyCopySource</Code><Message>The specified resource does not exist.
RequestId:fe725383-701e-0002-5aae-ccdd02000000
Time:2018-04-05T07:20:26.8836489Z</Message></Error>
I started to check the possible reasons but finally I registered an issue on Azure/azure-cli on Git to MS: az storage blob copy start issue when I use “sas token”
Some days later I received this answer:
@the1bit Thanks for bringing this to our attention.
#6041 will apply the sas token specified by –sas-token for the source as well as the destination and will be available in our next release.
For now, please use –source-sas to apply the same sas towards your source, as –sas-token currently only applies towards the destination.
I tested again then I was sure there is a bug in the code. I used this command:
az storage blob copy start --account-name mystorageacc --sas-token "?sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************" --destination-container "files" --source-container "files" --source-blob "new/arm-template.json" --destination-blob "archive/arm-template.json" --source-account-name mystorageacc --source-sas "?sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************" --debug
And the error was a new one:
AzureHttpError: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.ErrorCode: CannotVerifyCopySource <?xml version="1.0" encoding="utf-8"?><Error><Code>CannotVerifyCopySource</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
Then I found a strange thing in debug output:
'x-ms-copy-source': 'https://mystorageacc.blob.core.cloudapi.de/files/new/arm-template.json??sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2118-04-10T16:23:59Z&st=2018-04-10T08:23:59Z&spr=https&sig=4dCQpoGDZnHZY%2FCk0TXKXtH6I%2BzZP%2BTW2ZkErE1LgjQ%3D',
As you can see there is a double ‘?’ around SAS token:
.json??sv=
Then I tested the –source-sas without ‘?’:
--source-sas "sv=2017-07-29&ss=bfqt&srt=sco&sp=rwdlacup&se=2019-04-04T15:43:14Z&st=2018-04-03T23:43:14Z&spr=https&sig=***************"
and the copy works:
{
"completionTime": null,
"id": "077b06d7-d843-4fc3-ba8b-e48091753869",
"progress": null,
"source": null,
"status": "success",
"statusDescription": null
}
I sent this to MS and some another days later:
@the1bit I’ve raised a new issue for the bug you found: #6073
Thanks for finding this!
Workaround
There is a bug in “az storage blob copy start” with “–source-sas” parameter in azure-cli 2.0.30. I am sure they will fix this soon. MEanwhile you can apply the following workarounds:
- use your sas token in “–source-sas” parameter without ‘?’
- use storage account key instead of sas token.
I hope this helps to avoid some struggling until the fix will be here.
