auslib.blobs

auslib.blobs.apprelease

class auslib.blobs.apprelease.DesupportBlob(**kwargs)

Bases: auslib.blobs.base.Blob

This blob is used to inform users that their OS is no longer supported. This is available on the client side since Firefox 24 (bug 843497).

The XML should look like this (whitespace for clarity & consistency only):

<?xml version=”1.0”?> <updates>

<update type=”major” unsupported=”true” detailsURL=”http://moreinfo”> </update>

</updates>

containsForbiddenDomain(product, whitelistedDomains)
getInnerFooterXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerHeaderXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
jsonschema = 'desupport.yml'
shouldServeUpdate(updateQuery)

Should be implemented by subclasses. In the event that it’s not, False is the safest thing to return (it will fail closed instead of failing open).

class auslib.blobs.apprelease.MultipleUpdatesXMLMixin

Bases: object

class auslib.blobs.apprelease.NewStyleVersionsMixin

Bases: object

getAppVersion(platform, locale)
getApplicationVersion(platform, locale)

For v2 schema, appVersion really is the app version

getDisplayVersion(platform, locale)
getPlatformVersion(platform, locale)
class auslib.blobs.apprelease.ProofXMLMixin

Bases: object

class auslib.blobs.apprelease.ReleaseBlobBase(**kwargs)

Bases: auslib.blobs.base.Blob

containsForbiddenDomain(product, whitelistedDomains)

Returns True if the blob contains any file URLs that contain a domain that we’re not allowed to serve updates to.

getBuildID(platform, locale)
getInnerFooterXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerHeaderXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)

This method, along with getHeaderXML and getFooterXML are the entry point for update XML creation for all Gecko app blobs. However, the XML and underlying data has changed over time, so there is a lot of indirection and calls factored out to subclasses. Below is a brief description of the flow of control that should help in understanding this code. Inner methods that are shared between blob versions live in Mixin classes so that they can be easily shared. Inner methods that only apply to a single blob version live on concrete blob classes (but should be moved if they need to be shared in the future). * getInnerXML, getFooterXML and getHeaderXML called by web layer,

live on this base class. The V1 blob class override them to support bug 1113475, but still calls the base class one to do most of the work.

** _getUpdateLineXML() called to get information that is independent

of specific MARs. Most notably, version information changed starting with V2 blobs.

** _getPatchesXML() called to get the information that describes

specific MARs. Where in the blob this information comes from changed significantly starting with V3 blobs.

*** _getSpecificPatchXML() called to translate MAR information into

XML. This transformation in blob version independent, so it lives on the base class to avoid duplication.

**** _getUrl() called to figure out what the MAR URL is for a

specific patch. This changed starting with V4 blobs. V3 and earlier use SeparatedFileUrlsMixin, V4 and later use UnifiedFileUrlsMixin.

* _getFtpFilename/_getBouncerProduct called to substitute some

paths with real information. This is another part of the blob format that changed starting with V3 blobs. It was later deprecated in V4 and thus not used for UnifiedFileUrlsMixin blobs.

getLocaleData(platform, locale)
getLocaleOrTopLevelParam(platform, locale, param)
getPlatformData(platform)
getResolvedPlatform(platform)
matchesUpdateQuery(updateQuery)
shouldServeUpdate(updateQuery)

Should be implemented by subclasses. In the event that it’s not, False is the safest thing to return (it will fail closed instead of failing open).

class auslib.blobs.apprelease.ReleaseBlobV1(**kwargs)

Bases: auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.SingleUpdateXMLMixin, auslib.blobs.apprelease.SeparatedFileUrlsMixin

This is the legacy format for apps based on Gecko 1.8.0 to 1.9.2, which translates to

  • Firefox 1.5 to 3.6.x

  • Thunderbird 1.5 to 3.1.y

It was deprecated by https://bugzilla.mozilla.org/show_bug.cgi?id=530872 during Gecko 2.0 development (aka 1.9.3).

createSnippets(updateQuery, update_type, whitelistedDomains, specialForceHosts)
getApplicationVersion(platform, locale)

We used extv as the application version for v1 schema, while appv may have been a pretty version for users to see

getAppv(platform, locale)
getExtv(platform, locale)
getInnerHeaderXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)

In order to update some older versions of Firefox without prompting them for add-on compatibility, we need to be able to modify the appVersion and extVersion attributes. bug 998721 and bug 1174605 have additional background on this. It would be nicer to do this in _getUpdateLineXML to avoid overriding this method, but that doesn’t have access to the updateQuery to lookup the version making the request.

getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

jsonschema = 'apprelease-v1.yml'
class auslib.blobs.apprelease.ReleaseBlobV2(**kwargs)

Bases: auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.NewStyleVersionsMixin, auslib.blobs.apprelease.SingleUpdateXMLMixin, auslib.blobs.apprelease.SeparatedFileUrlsMixin

Compatible with Gecko 1.9.3a3 and above, ie Firefox/Thunderbird 4.0 and above.

Client-side changes in

https://bugzilla.mozilla.org/show_bug.cgi?id=530872

renamed or introduced several attributes in update.xml

Changed parameters from ReleaseBlobV1:
  • appv, extv become appVersion, platformVersion, displayVersion

Added:
  • actions, billboardURL, openURL, notificationURL, alertURL, showPrompt, showNeverForVersion, isOSUpdate

Removed:
  • oldVersionSpecialCases

createSnippets(updateQuery, update_type, whitelistedDomains, specialForceHosts)
getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

interpolable_ = ('billboardURL', 'openURL', 'notificationURL', 'alertURL')
jsonschema = 'apprelease-v2.yml'
optional_ = ('billboardURL', 'showPrompt', 'showNeverForVersion', 'actions', 'openURL', 'notificationURL', 'alertURL')
class auslib.blobs.apprelease.ReleaseBlobV3(**kwargs)

Bases: auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.NewStyleVersionsMixin, auslib.blobs.apprelease.MultipleUpdatesXMLMixin, auslib.blobs.apprelease.SeparatedFileUrlsMixin

Compatible with Gecko 1.9.3a3 and above, ie Firefox/Thunderbird 4.0 and above.

This is an internal change to add functionality to Balrog.

Changes from ReleaseBlobV2:
  • support multiple partials * remove “partial” and “complete” from locale level * add “partials” and “completes” to locale level, ftpFilenames, and bouncerProducts

createSnippets(updateQuery, update_type, whitelistedDomains, specialForceHosts)
getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

interpolable_ = ('billboardURL', 'openURL', 'notificationURL', 'alertURL')
jsonschema = 'apprelease-v3.yml'
optional_ = ('billboardURL', 'showPrompt', 'showNeverForVersion', 'actions', 'openURL', 'notificationURL', 'alertURL')
class auslib.blobs.apprelease.ReleaseBlobV4(**kwargs)

Bases: auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.NewStyleVersionsMixin, auslib.blobs.apprelease.MultipleUpdatesXMLMixin, auslib.blobs.apprelease.UnifiedFileUrlsMixin

Compatible with Gecko 1.9.3a3 and above, ie Firefox/Thunderbird 4.0 and above.

This is an internal change to add functionality to Balrog.

Changes from ReleaseBlobV3:
  • Support pushing release builds to the beta channel with bouncer support (bug 1021026)

** Combine fileUrls, bouncerProducts, and ftpFilenames into a larger data structure,

still called “fileUrls”. (See below for a more detailed description.)

classmethod fromV3(v3Blob)

Creates a v4 blob based on the v3 blob given.

getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

interpolable_ = ('billboardURL', 'openURL', 'notificationURL', 'alertURL')
jsonschema = 'apprelease-v4.yml'
optional_ = ('billboardURL', 'showPrompt', 'showNeverForVersion', 'actions', 'openURL', 'notificationURL', 'alertURL')
class auslib.blobs.apprelease.ReleaseBlobV5(**kwargs)

Bases: auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.NewStyleVersionsMixin, auslib.blobs.apprelease.MultipleUpdatesXMLMixin, auslib.blobs.apprelease.UnifiedFileUrlsMixin

Compatible with Gecko 19.0 and above, ie Firefox/Thunderbird 19.0 and above.

Driven by a client-side change made in

https://bugzilla.mozilla.org/show_bug.cgi?id=813322

Changes from ReleaseBlobV4:
  • Support optional promptWaitTime attribute

getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

interpolable_ = ('billboardURL', 'openURL', 'notificationURL', 'alertURL')
jsonschema = 'apprelease-v5.yml'
optional_ = ('billboardURL', 'showPrompt', 'showNeverForVersion', 'actions', 'openURL', 'notificationURL', 'alertURL', 'promptWaitTime')
class auslib.blobs.apprelease.ReleaseBlobV6(**kwargs)

Bases: auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.NewStyleVersionsMixin, auslib.blobs.apprelease.MultipleUpdatesXMLMixin, auslib.blobs.apprelease.UnifiedFileUrlsMixin

Compatible with Gecko 51.0 and above, ie Firefox/Thunderbird 51.0 and above.

Changes from ReleaseBlobV5:
  • Removes support for platformVersion, billboardURL, licenseURL, version, and extensionVersion

For further information:
getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

interpolable_ = ('openURL', 'notificationURL', 'alertURL')
jsonschema = 'apprelease-v6.yml'
optional_ = ('showPrompt', 'showNeverForVersion', 'actions', 'openURL', 'notificationURL', 'alertURL', 'promptWaitTime')
class auslib.blobs.apprelease.ReleaseBlobV8(**kwargs)

Bases: auslib.blobs.apprelease.ProofXMLMixin, auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.NewStyleVersionsMixin, auslib.blobs.apprelease.MultipleUpdatesXMLMixin, auslib.blobs.apprelease.UnifiedFileUrlsMixin

Compatible with Gecko 51.0 and above, ie Firefox/Thunderbird 51.0 and above.

Changes from ReleaseBlobV6:
  • Adds support for ProofXMLMixin (placed as first parameter for inheritance preference)

For further information:
getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

interpolable_ = ('openURL', 'notificationURL', 'alertURL')
jsonschema = 'apprelease-v8.yml'
optional_ = ('showPrompt', 'showNeverForVersion', 'actions', 'openURL', 'notificationURL', 'alertURL', 'promptWaitTime', 'binTransMerkleRoot', 'binTransCertificate', 'binTransSCTList')
class auslib.blobs.apprelease.ReleaseBlobV9(**kwargs)

Bases: auslib.blobs.apprelease.ProofXMLMixin, auslib.blobs.apprelease.ReleaseBlobBase, auslib.blobs.apprelease.MultipleUpdatesXMLMixin, auslib.blobs.apprelease.UnifiedFileUrlsMixin

Compatible with Gecko 51.0 and above, ie Firefox/Thunderbird 51.0 and above.

Changes from ReleaseBlobV8:
  • Moved most <update> properties into new updateLine data structure

For further information:
getApplicationVersion(platform, locale)
getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

interpolable_ = ('openURL', 'notificationURL', 'alertURL', 'detailsURL')
jsonschema = 'apprelease-v9.yml'
validate(*args, **kwargs)

Raises a BlobValidationError if the blob is invalid.

class auslib.blobs.apprelease.SeparatedFileUrlsMixin

Bases: object

class auslib.blobs.apprelease.SingleUpdateXMLMixin

Bases: object

class auslib.blobs.apprelease.UnifiedFileUrlsMixin

Bases: object

auslib.blobs.base

class auslib.blobs.base.Blob(*args, **kwargs)

Bases: dict

containsForbiddenDomain(product, whitelistedDomains)
getFooterXML()
Returns

Returns the outer most footer. Returns the outer most header

getHeaderXML()
Returns

Returns the outer most header. Returns the outer most header

getInnerFooterXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerHeaderXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
getJSON()

Returns a JSON formatted version of this blob.

getReferencedReleases()
Returns

Returns set of names of partially referenced releases that the current

release references

getResponseBlobs()
Returns

Usually returns None. It the Blob is a systemaddons superblob, it returns the list of return blobs

getResponseProducts()
Returns

Usually returns None. If the Blob is a SuperBlob, it returns the list of return products.

getSchema()
jsonschema = None
loadJSON(data)

Replaces this blob’s contents with parsed contents of the json string provided.

processSpecialForceHosts(url, specialForceHosts, force_arg)
shouldServeUpdate(updateQuery)

Should be implemented by subclasses. In the event that it’s not, False is the safest thing to return (it will fail closed instead of failing open).

validate(product, whitelistedDomains)

Raises a BlobValidationError if the blob is invalid.

exception auslib.blobs.base.BlobValidationError(message, errors, *args, **kwargs)

Bases: ValueError

auslib.blobs.base.createBlob(data)

Takes a string form of a blob (eg from DB or API) and converts into an actual blob, taking care to notice the schema

auslib.blobs.base.merge_dicts(ancestor, left, right)

Perform a 3-way merge on dictonaries. We used to use an external library for this, but we replaced it with this because our merge can be a bit more liberal than the general case. Specifically:

  • Lists are treated as sets (see above for details)

  • A type mismatch of unicode vs string is OK as long as the text is the same (Any other type mismatches result in a failure to merge.)

auslib.blobs.base.merge_lists(*lists)

Merge an arbitrary number of lists together, keeping only the unique items, and not worrying about order. This is essentially a hack to treat lists in blobs as sets. In an ideal world, that’s what they’d be, but because we use jsonschema for validation, we cannot use proper sets.

auslib.blobs.gmp

class auslib.blobs.gmp.GMPBlobV1(**kwargs)

Bases: auslib.blobs.base.Blob

containsForbiddenDomain(product, whitelistedDomains)

Returns True if the blob contains any file URLs that contain a domain that we’re not allowed to serve updates to.

getInnerFooterXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerHeaderXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
getPlatformData(vendor, platform)
getResolvedPlatform(vendor, platform)
getVendorsForPlatform(platform)
jsonschema = 'gmp.yml'
shouldServeUpdate(updateQuery)

Should be implemented by subclasses. In the event that it’s not, False is the safest thing to return (it will fail closed instead of failing open).

validate(product, whitelistedDomains)

Raises a BlobValidationError if the blob is invalid.

auslib.blobs.superblob

class auslib.blobs.superblob.SuperBlob(**kwargs)

Bases: auslib.blobs.base.Blob

containsForbiddenDomain(product, whitelistedDomains)
getInnerFooterXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerHeaderXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Header specific to GMP and systemaddons superblob

getResponseBlobs()
Returns

Blob names in case of systemaddon Blobs

getResponseProducts()
Returns

Product in case of GMP supreblob

jsonschema = 'superblob.yml'
shouldServeUpdate(updateQuery)

Should be implemented by subclasses. In the event that it’s not, False is the safest thing to return (it will fail closed instead of failing open).

auslib.blobs.systemaddons

class auslib.blobs.systemaddons.SystemAddonsBlob(**kwargs)

Bases: auslib.blobs.base.Blob

containsForbiddenDomain(product, whitelistedDomains)

Returns True if the blob contains any file URLs that contain a domain that we’re not allowed to serve updates to.

getAddonsForPlatform(platform)
getInnerFooterXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerHeaderXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
Returns

Releases-specific header should be implemented for individual blobs

getInnerXML(updateQuery, update_type, whitelistedDomains, specialForceHosts)
getPlatformData(addon, platform)
getResolvedPlatform(addon, platform)
hasUpdates(updateQuery, whitelistedDomains)
jsonschema = 'systemaddons.yml'
shouldServeUpdate(updateQuery)

Should be implemented by subclasses. In the event that it’s not, False is the safest thing to return (it will fail closed instead of failing open).

auslib.blobs.whitelist