ALiSNet Model Documentation¶
Model Owner: Amrollah Seifoddini
Document Version: 0.2.0
Reviewers: Alex Loosley, Rocco Maresca
Model documentation contributions and feedback are welcome!
Overview¶
Model Type¶
Convolutional Neural Network
Model Description¶
Accurate and Lightweight mobile human Segmentation Network (ALiSNet) is a Convolutional Neural Network (CNN) for semantic segmentation of human silhouettes from photos. Architecture of ALiSNet is based on Semantic FPN with PointRend refinement. The original backbone of the network is exchanged by mobile-optimized MnasNet to achieve significant model size reduction. Additionaly, Feature Pyramid Network (FPN) from the original backbone is replaced with a simpler aggregation step, skipping FPN top-down path. Final model is also quantized to 8 bits integer precision using Qantization Aware Training.
Status¶
Status Date: 2025-01-22
Regularly Updated - New versions of the model have been or will continue to be made available.
Relevant Links¶
Developer(s)¶
Amrollah Seifoddini, Size & Fit
Owner(s)¶
Team Name: Size & Fit
Contact Person: Amrollah Seifoddini
Version Details and Artifacts¶
Current Model Version: Not currently tracked
Model Version Release Date: N/A
Model Version for last Model Card Update: N/A
Artifacts:
Intended and Known Usage¶
Intended Use¶
Model was developed to perform semantic segmentation of humans vs. background as a step in inference of body measurements from photos. Given the specific use case, ALiSNet was trained to work for very narrow arm and leg angles ranges.
Out of Scope Uses¶
ALiSNet was trained to work for very narrow arm and leg angles ranges. It shouldn't be used for general semantic segmentation tasks of human silhouettes vs. background.
Known Applications¶
| Application | Purpose of Model Usage | AI Act Risk |
|---|---|---|
| Body Measurements Pipeline | ALiSNet is used for silhouette extraction as part of Size & Fit's Body Measurements Pipeline. | Limited |
Note, this table may not be exhaustive. Model users and documentation consumers at large are highly encouraged to contribute known usages.
Risks and Ethical Considerations¶
Privacy risks were initially reviewed by Size & Fit (J. Peterman, Frank Constanza, David Puddy, Stefan Messmer) in collaboration with Algorithmic Privacy & Fairness (Elaine Benes) and Product Security (Kenny Bania).
Processing or storage of customer photos during deployment¶
In order to obtain measurments customers need to upload their photos to our app so that their silhouette may be inferred. This creates a risk related to storage and processing of private data.
Mitigation strategy:
ALiSNet is deployed on-device. Customers photos are processed and stored on their device and Zalando does not have access to them. Only inferred silhouettes are sent to Zalando servers.
On-device deployment¶
Deploying ALiSNet on-device may cause risks related to backdoor attacks against ML models such as Membership Inference or Model Inversion which may result in violation of privacy of our customers.
Mitigation strategy:
Privacy risk assessment was performed in order to quantify Membership Inference (MIA) risk of the ALiSNet. As of November 2022 results of the audit showed that MIA risk is none to low.
Biases in model performance¶
Statistically relevant discrepancies in the model accuracy across protected attributes related to human characteristics such as age, percieved skin tone, height etc..
Mitigation Strategy:
Fairnes assessment was performed in order to qunatify potential discerpancies across derived skin tone and body shape. As of October 2022 no relevant discrepencies in model performance were found.
Training¶
ALiSNet is a classification model trained on semantic segmentation task which goal is to perform a classification of pixels in the input image with corresponding semantic class (human, bike etc.).
See ALiSNet Paper for more details about training.
Datasets¶
| Name | Location | Sensitive* | Size |
|---|---|---|---|
| Filtered Common Objects in Context (COCO) | s3://sagemaker-io-2/coco/ | No* | ~50k samples |
| Another Production Dataset Training Split (Another Dataset Train) | s3://obfuscated-bucket-name/production/ | Yes* | 3625 samples |
(*): Requires a special Data Processing Request
Evaluation¶
See ALiSNet Paper for more details about testing such as performance metrics and baselines.
Performance Metrics¶
Metrics used to evaluate model:
Results:
- Voices of Customers (VOC):
- MIoU: 97.6 (+/- 0.1)
Model Bias and Fairness Analysis¶
See Body Measurement Prediction Fairness Paper for complete fairness assessment results.
Datasets¶
ALiSNet was evaluated using multiple datasets. All of the datasets represent realistic examples expected to be seen in intended deployment scenario.
| Name | Location | Sensitive* | Size |
|---|---|---|---|
| Faces dataset1 | s3://obfuscated-bucket-name/faces/production/ | Yes* | 2416 samples |
| Voices of Customers (VOC) dataset1,2 | s3://obfuscated-bucket-name/voc/production/ | Yes* | 6494 samples |
| Optimass dataset1 | s3://obfuscated-bucket-name/optimass/production/ | Yes* | 322 samples |
| Skin tones dataset2 | s3://obfuscated-bucket-name/skin-tones/production/ | Yes* | 499 samples |
(*): Requires a special Data Processing Request
(1): Used for overall performance evaluation
(2): Used for fairness evaluation
Caveats and Recommendations¶
Poor lightning conditions and low photo resolution may decrease quality of segmentation. (?)
The model was designed with strong size and latency constraints and is ready to be deployed to production or edge devices.
Documentation Metadata¶
Note, some names in this document have been anonymized.
Documentation Template Version¶
Documentation Authors¶
- Rocco Maresca, Algorithmic Privacy & Fairness: (Owner)