Merge pull request johnbeard#5 from INTI-CMNB/variants

Added Variants and Filters

Author: set-soft

CHANGELOG.md

@@ -6,6 +6,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 
 ## [Unreleased]
+### Added
+- Now variants are separated entities.
+  - Only the internal BoM currently supports it.
+  - In the future IBoM will also support it, contact me if you think this is
+    high priority.
+- New filters entities. They implement all the functionality in KiBoM and IBoM.
+- Implemented the IBoM variants style.
+- The internal BoM format supports filters for:
+  - Excluding components from processing
+  - Marking components as "Do Not Fit"
+  - Marking components as "Do Not Change"
+- The internal BoM format supports KiBoM and IBoM style variants
 
 ## [0.6.2] - 2020-08-25
 ### Changed

README.md

@@ -7,5 +7,5 @@ all: ../README.md  samples/generic_plot.kibot.yaml
 
 samples/generic_plot.kibot.yaml: ../kibot/out_*.py ../kibot/pre_*.py ../kibot/config_reader.py
 	rm -f example.kibot.yaml
-	../src/kibot --example
+	../src/kibot -v --example
 	mv example.kibot.yaml $@

docs/Makefile

@@ -63,36 +63,16 @@ outputs:
         quote_all: false
         # [string=','] CSV Separator. TXT and TSV always use tab as delimiter
         separator: ','
-      # [list(dict)] A series of regular expressions used to exclude parts.
-      # If a component matches ANY of these, it will be excluded.
-      # Column names are case-insensitive.
-      # If empty the following list is used:
-      # - column: References
-      #   regex: '^TP[0-9]*'
-      # - column: References
-      #   regex: '^FID'
-      # - column: Part
-      #   regex: 'mount.*hole'
-      # - column: Part
-      #   regex: 'solder.*bridge'
-      # - column: Part
-      #   regex: 'solder.*jump'
-      # - column: Part
-      #   regex: 'test.*point'
-      # - column: Footprint
-      #   regex: 'test.*point'
-      # - column: Footprint
-      #   regex: 'mount.*hole'
-      # - column: Footprint
-      #   regex: 'fiducial'
-      exclude_any:
-        # [string=''] Name of the column to apply the regular expression
-        column: ''
-        # `field` is an alias for `column`
-        # [string=''] Regular expression to match
-        regex: ''
-        # `regexp` is an alias for `regex`
-      # [string='Config'] Field name used to determine if a particular part is to be fitted (also DNC and variants)
+      # [string|list(string)='_kibom_dnc'] Name of the filter to mark components as 'Do Not Change'.
+      # The default filter marks components with a DNC value or DNC in the Config field
+      dnc_filter: '_kibom_dnc'
+      # [string|list(string)='_kibom_dnf'] Name of the filter to mark components as 'Do Not Fit'.
+      # The default filter marks components with a DNF value or DNF in the Config field
+      dnf_filter: '_kibom_dnf'
+      # [string|list(string)='_mechanical'] Name of the filter to exclude components from BoM processing.
+      # The default filter excludes test points, fiducial marks, mounting holes, etc
+      exclude_filter: '_mechanical'
+      # [string='Config'] Field name used for internal filters
       fit_field: 'Config'
       # [string=''] [HTML,CSV,TXT,TSV,XML,XLSX] format for the BoM.
       # If empty defaults to CSV or a guess according to the options.
@@ -129,17 +109,6 @@ outputs:
         title: 'KiBot Bill of Materials'
       # [boolean=true] Exclude DNF (Do Not Fit) components
       ignore_dnf: true
-      # [list(dict)] A series of regular expressions used to select included parts.
-      # If there are any regex defined here, only components that match against ANY of them will be included.
-      # Column names are case-insensitive.
-      # If empty all the components are included
-      include_only:
-        # [string=''] Name of the column to apply the regular expression
-        column: ''
-        # `field` is an alias for `column`
-        # [string=''] Regular expression to match
-        regex: ''
-        # `regexp` is an alias for `regex`
       # [boolean=true] Component groups with blank fields will be merged into the most compatible group, where possible
       merge_blank_fields: true
       # [boolean=false] When normalizing values use the locale decimal point
@@ -148,14 +117,11 @@ outputs:
       normalize_values: false
       # [number=1] Number of boards to build (components multiplier)
       number: 1
-      # [string='%f-%i.%x'] filename for the output (%i=bom). Affected by global options
-      output: '%f-%i.%x'
-      # [boolean=true] Each component group will be tested against a number of regular-expressions
-      # (see `include_only` and `exclude_any`)
-      test_regex: true
+      # [string='%f-%i%v.%x'] filename for the output (%i=bom). Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=false] Print grouped references in the alternate compressed style eg: R1-R7,R18
       use_alt: false
-      # [string|list(string)=''] Board variant(s), used to determine which components
+      # [string=''] Board variant(s), used to determine which components
       # are output to the BoM.
       variant: ''
       # [dict] Options for the XLSX format
@@ -200,8 +166,8 @@ outputs:
       force_plot_invisible_refs_vals: false
       # [boolean=false] use mm instead of inches
       metric_units: false
-      # [string='%f-%i.%x'] output file name, the default KiCad name if empty. Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] output file name, the default KiCad name if empty. Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=true] include the footprint references
       plot_footprint_refs: true
       # [boolean=true] include the footprint values
@@ -229,8 +195,8 @@ outputs:
       # [dict|string] [hpgl,ps,gerber,dxf,svg,pdf] format for a graphical drill map.
       # Not generated unless a format is specified
       map:
-        # [string='%f-%i.%x'] name for the map file, KiCad defaults if empty (%i='PTH_drill_map'). Affected by global options
-        output: '%f-%i.%x'
+        # [string='%f-%i%v.%x'] name for the map file, KiCad defaults if empty (%i='PTH_drill_map'). Affected by global options
+        output: '%f-%i%v.%x'
         # [string='pdf'] [hpgl,ps,gerber,dxf,svg,pdf] format for a graphical drill map
         type: 'pdf'
       # [boolean=true] use metric units instead of inches
@@ -239,8 +205,8 @@ outputs:
       minimal_header: false
       # [boolean=false] invert the Y axis
       mirror_y_axis: false
-      # [string='%f-%i.%x'] name for the drill file, KiCad defaults if empty (%i='PTH_drill'). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] name for the drill file, KiCad defaults if empty (%i='PTH_drill'). Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=true] generate one file for both, plated holes and non-plated holes, instead of two separated files
       pth_and_npth_single_file: true
       # [dict|string] name of the drill report. Not generated unless a name is specified
@@ -262,12 +228,12 @@ outputs:
       # [dict|string] [hpgl,ps,gerber,dxf,svg,pdf] format for a graphical drill map.
       # Not generated unless a format is specified
       map:
-        # [string='%f-%i.%x'] name for the map file, KiCad defaults if empty (%i='PTH_drill_map'). Affected by global options
-        output: '%f-%i.%x'
+        # [string='%f-%i%v.%x'] name for the map file, KiCad defaults if empty (%i='PTH_drill_map'). Affected by global options
+        output: '%f-%i%v.%x'
         # [string='pdf'] [hpgl,ps,gerber,dxf,svg,pdf] format for a graphical drill map
         type: 'pdf'
-      # [string='%f-%i.%x'] name for the drill file, KiCad defaults if empty (%i='PTH_drill'). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] name for the drill file, KiCad defaults if empty (%i='PTH_drill'). Affected by global options
+      output: '%f-%i%v.%x'
       # [dict|string] name of the drill report. Not generated unless a name is specified
       report:
         # [string=''] name of the drill report. Not generated unless a name is specified.
@@ -298,8 +264,8 @@ outputs:
       gerber_precision: 4.6
       # [number=0.1] [0.02,2] line_width for objects without width [mm]
       line_width: 0.1
-      # [string='%f-%i.%x'] output file name, the default KiCad name if empty. Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] output file name, the default KiCad name if empty. Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=true] include the footprint references
       plot_footprint_refs: true
       # [boolean=true] include the footprint values
@@ -337,8 +303,8 @@ outputs:
       force_plot_invisible_refs_vals: false
       # [boolean=false] plot mirrored
       mirror_plot: false
-      # [string='%f-%i.%x'] output file name, the default KiCad name if empty. Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] output file name, the default KiCad name if empty. Affected by global options
+      output: '%f-%i%v.%x'
       # [number=1] [1,16] pen number
       pen_number: 1
       # [number=20] [1,99] pen speed
@@ -415,8 +381,8 @@ outputs:
       no_redraw_on_drag: false
       # [boolean=false] Normalize extra field name case. E.g. 'MPN' and 'mpn' will be considered the same field
       normalize_field_case: false
-      # [string='%f-%i.%x'] Filename for the output, use '' to use the IBoM filename (%i=ibom, %x=html). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] Filename for the output, use '' to use the IBoM filename (%i=ibom, %x=html). Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=false] Show fabrication layer by default
       show_fabrication: false
       # [string='C,R,L,D,U,Y,X,F,SW,A,~,HS,CNN,J,P,NT,MH'] Default sort order for components. Must contain '~' once
@@ -530,8 +496,8 @@ outputs:
       format: 'HTML'
       # [number=1] Number of boards to build (components multiplier)
       number: 1
-      # [string='%f-%i.%x'] filename for the output (%i=bom). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] filename for the output (%i=bom). Affected by global options
+      output: '%f-%i%v.%x'
       # [string=','] CSV Separator
       separator: ','
       # [string=''] Board variant(s), used to determine which components
@@ -562,8 +528,8 @@ outputs:
       mirror: false
       # [boolean=false] do not make holes transparent
       no_drillholes: false
-      # [string='%f-%i.%x'] name for the generated file. Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] name for the generated file. Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=false] show placeholder for missing components
       placeholder: false
       # [dict|None] replacements for PCB references using components (lib:component)
@@ -620,8 +586,8 @@ outputs:
       mirror_plot: false
       # [boolean=false] invert black and white
       negative_plot: false
-      # [string='%f-%i.%x'] output file name, the default KiCad name if empty. Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] output file name, the default KiCad name if empty. Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=true] include the footprint references
       plot_footprint_refs: true
       # [boolean=true] include the footprint values
@@ -640,8 +606,8 @@ outputs:
     type: 'pdf_pcb_print'
     dir: 'Example/pdf_pcb_print_dir'
     options:
-      # [string='%f-%i.%x'] filename for the output PDF (%i=layers, %x=pdf). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] filename for the output PDF (%i=layers, %x=pdf). Affected by global options
+      output: '%f-%i%v.%x'
       # `output_name` is an alias for `output`
     layers: all
 
@@ -653,8 +619,8 @@ outputs:
     type: 'pdf_sch_print'
     dir: 'Example/pdf_sch_print_dir'
     options:
-      # [string='%f-%i.%x'] filename for the output PDF (%i=schematic %x=pdf). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] filename for the output PDF (%i=schematic %x=pdf). Affected by global options
+      output: '%f-%i%v.%x'
 
   # Pick & place:
   # This output is what you get from the 'File/Fabrication output/Footprint poistion (.pos) file' menu in pcbnew.
@@ -667,8 +633,8 @@ outputs:
       format: 'ASCII'
       # [boolean=true] only include the surface mount components
       only_smd: true
-      # [string='%f-%i.%x'] output file name (%i='top_pos'|'bottom_pos'|'both_pos', %x='pos'|'csv'). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] output file name (%i='top_pos'|'bottom_pos'|'both_pos', %x='pos'|'csv'). Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=true] generate two separated files, one for the top and another for the bottom
       separate_files_for_front_and_back: true
       # [string='millimeters'] [millimeters,inches] units used for the positions
@@ -697,8 +663,8 @@ outputs:
       mirror_plot: false
       # [boolean=false] invert black and white
       negative_plot: false
-      # [string='%f-%i.%x'] output file name, the default KiCad name if empty. Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] output file name, the default KiCad name if empty. Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=true] include the footprint references
       plot_footprint_refs: true
       # [boolean=true] include the footprint values
@@ -738,8 +704,8 @@ outputs:
       # The drill option uses the auxiliar reference defined by the user.
       # You can define any other origin using the format 'X,Y', i.e. '3.2,-10'
       origin: 'grid'
-      # [string='%f-%i.%x'] name for the generated STEP file (%i='3D' %x='step'). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] name for the generated STEP file (%i='3D' %x='step'). Affected by global options
+      output: '%f-%i%v.%x'
 
   # SVG (Scalable Vector Graphics):
   # Unlike bitmaps SVG drawings can be scaled without losing resolution.
@@ -763,8 +729,8 @@ outputs:
       mirror_plot: false
       # [boolean=false] invert black and white
       negative_plot: false
-      # [string='%f-%i.%x'] output file name, the default KiCad name if empty. Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] output file name, the default KiCad name if empty. Affected by global options
+      output: '%f-%i%v.%x'
       # [boolean=true] include the footprint references
       plot_footprint_refs: true
       # [boolean=true] include the footprint values
@@ -782,6 +748,6 @@ outputs:
     type: 'svg_sch_print'
     dir: 'Example/svg_sch_print_dir'
     options:
-      # [string='%f-%i.%x'] filename for the output SVG (%i=schematic %x=svg). Affected by global options
-      output: '%f-%i.%x'
+      # [string='%f-%i%v.%x'] filename for the output SVG (%i=schematic %x=svg). Affected by global options
+      output: '%f-%i%v.%x'
 

docs/samples/generic_plot.kibot.yaml

@@ -0,0 +1,35 @@
+# IBoM variants
+
+This is an analysis and test of the *variants* implementation of [IBoM](https://github.com/openscopeproject/InteractiveHtmlBom)
+
+## What goes inside the SCH
+
+- The field used for variants must be specified using `--variant-field`
+- The field can contain only one value. So you create some kind of component groups.
+
+## What goes outside the SCH
+
+- Two optional lists are passed to create the variant.
+- Components without a group are always included.
+- Whitelist: only the groups listed here are included.
+  - If this list is empty all groups are included, unless listed in the blacklist.
+- Blacklist: groups listed here are excluded.
+
+## Where is in the code?
+
+In core/ibom.py function skip_component.
+
+## Conclusion
+
+### Advantages
+
+- The `Config` field is simple.
+- You have "exclude from" and "include only" options.
+
+### Disadvantages
+
+- Critical part of the information is outside the project.
+
+
+
+

experiments/variants/IBoM/README.md

@@ -0,0 +1,37 @@
+# KiBOM variants
+
+This is an analysis and test of the *variants* implementation of [KiBOM](https://github.com/SchrodingersGat/KiBoM)
+
+## What goes inside the SCH
+
+- The variants are implemented using the `Config` field (the name can be configured).
+- By default a component is included in all variants.
+- You can exclude a component from one variant adding `-VARIANT_NAME` to the `Config`.
+  - If you want to exclude this component from more than one variant just add more `-VARIANT_NAME` entries. Comma separated.
+- If a component will be included **only** in one variant you can use `+VARIANT_NAME`.
+  - Again you can add more than one `+VARIANT_NAME` entry. So the component will be included only if generating one of the selected variants.
+
+## What goes outside the SCH
+
+- When you generate the BoM you can select one or more variants, again comma separated.
+- The `-VARIANT_NAME` and `+VARIANT_NAME` is tested using a list of all the indicated variants.
+- By default the list of variants is ['default']. So *default* is like a special variant.
+
+## Where is in the code?
+
+The Component.isFitted() method implements the functionality.
+
+## Conclusion
+
+### Advantages
+
+- Almost all the information is inside the project.
+- You have "exclude from" and "include only" options.
+
+### Disadvantages
+
+- The `Config` field could become large.
+
+
+
+