feat(azure): ensure CustomData is base64-decodable - #6958
Conversation
blackboxsw
left a comment
There was a problem hiding this comment.
@cadejacobson thank you for this. I'd like to move the scope of the try/except closer to where we actually expect that error to be raised instead of further up the stack. The other comment about message failure details is non-blocking.
|
|
||
| class ReportableErrorOvfInvalidBase64(ReportableError): | ||
| def __init__(self, field: str) -> None: | ||
| super().__init__(f"failure to decode ovf-env.xml field={field}") |
There was a problem hiding this comment.
It seems we are only reporting the field as supporting data but not the representation of the specific cause of the error.
binascii.Error has more detail that could aid in the debug of this issue.
str(error) in helpers/azure.py gives us Invalid base64-encoded string: number of data characters (5) cannot be 1 more than a multiple of 4 Do we want to include this detail in the raised ReportableError somehow?
There was a problem hiding this comment.
Perhaps this is non-blocking, and your intent is to provide a simple readable string to represent all classes of non-base64 encoded data without details. Just wanted to raise this for your thoughts before we proceed.
There was a problem hiding this comment.
Thanks for raising this! This message was initially supposed to be rather general in order to allow for some of our internal queries on deployment statistics to be able to gather the InvalidBase64 deployments into one group. This may be achievable while still providing the additional message in that error output.
Let me work with @cjp256 to figure out the best way to go from here and I will get back to you.
There was a problem hiding this comment.
Yeah I agree it would be good to include some additional details in supporting data like length and maybe name of error type caught. We just need to be careful to include safe strings so we don't accidentally leak data contents if contained in the thrown exception. I don't expect this to be the case but we would need to validate.
There was a problem hiding this comment.
Updated to include a bit more data in the exception string. It appears no secrets will be reported in the failure message. Here is an example:
[ 9.856048] cloud-init[689]: 2026-07-29 13:57:49,418 - azure.py[ERROR]: Azure datasource failure occurred: result=error|reason=failure to decode ovf-env.xml field=CustomData|agent=Cloud-Init|field=CustomData|'exception=Error(''Invalid base64-encoded string: number of data characters (9) cannot be 1 more than a multiple of 4'')'|length=10|vm_id={redacted}|timestamp=2026-08-04T13:35:14.359589+00:00|documentation_url=https://aka.ms/linuxprovisioningerror.The length being different is actually valuable, as I included a non-data character (_) in my mocked bad-base64. This difference helps to show that there are non-data characters getting stripped out of the base64 while decoding, and how many there are.
| decode_base64=True, | ||
| required=False, | ||
| ) | ||
| except binascii.Error as error: |
There was a problem hiding this comment.
since this error can only be raised when decode_base64=True, can we instead move this try/except scope down into _parse_property:around the single line 1087 instead of up in _parse_linux_configuration_set_section?
There was a problem hiding this comment.
This is much cleaner and is better practice. Thanks for noticing this! This should be fixed in the most recent commit.
| ), | ||
| ( | ||
| "not_base64", # bad character stripped away | ||
| 10, |
There was a problem hiding this comment.
TIL that validate=False ignores any invalid b64 character, not just whitespace.
There was a problem hiding this comment.
given this, I agree these are the only two cases that throw exceptions. Incorrect badding or invalid number of data characters. This test coverage provides additional confidence that it can't/won't leak the data contents in the error. I reviewed the binascii/base64 implementations and these are the only two error cases I see with altchars=None and validate=False.
|
|
||
| parsed = azure_helper.OvfEnvXml.parse_text(ovf) | ||
|
|
||
| assert parsed.custom_data is None |
There was a problem hiding this comment.
interesting that custom_data is None instead of ""?
There was a problem hiding this comment.
I believe that is due to ElementTree's parsing of empty tags, or beginning and closing tags one after the other: https://docs.python.org/3.12/library/xml.etree.elementtree.html#element-objects
Since the CustomData XML tag appears as <CustomData></CustomData>, the text is not ever updated from the default value of None. A similar thing happens at the following test that was merged last week:
https://github.com/canonical/cloud-init/pull/6948/changes#diff-6e31b416f1464a8d16e2519ac4b4ce66946b54100de33dc5d40b27265f44871dR1447
Proposed Commit Message
Test Steps
This can be validated by using a custom cloud-init version that passes a non-base64 CustomData to cloud-init. The following was done on an Azure VM with the following
not_base64string:This forced CustomData value resulted in the correct error propagating to Azure:
My test changes for this can be found on my personal fork at the linked branch.
Merge type
Fixes #6959