Cubes 0.6 to 0.10.2 Release Notes¶
0.10.2¶
Summary:
- many improvements in handling multiple hierarchies
- more support of multiple hierarchies in the slicer server either as
parameter or with syntax
dimension@hierarchy
:- dimension values:
GET /dimension/date?hierarchy=dqmy
- cut: get first quarter of 2012
?cut=date@dqmy:2012,1
- drill-down on hierarchy with week on implicit (next) level:
?drilldown=date@ywd
- drill-down on hierarchy with week with exlpicitly specified week level:
?drilldown=date@ywd:week
- dimension values:
- order and order attribute can now be specified for a Level
- optional safe column aliases (see docs for more info) for databases that have non-standard requirements for column labels even when quoted
Thanks:
- Jose Juan Montes (@jjmontesl)
- Andrew Zeneski
- Reinier Reisy Quevedo Batista (@rquevedo)
New Features¶
- added order to Level object - can be
asc
,desc
or None for unspecified order (will be ignored) - added order_attribute to Level object - specifies attribute to be used for ordering according to order. If not specified, then first attribute is going to be used.
- added hierarchy argument to AggregationResult.table_rows()
- str(cube) returns cube name, useful in functions that can accept both cube name and cube object
- added cross table formatter and its HTML variant
GET /dimension
accepts hierarchy parameter- added create_workspace_from_config() to simplify workspace creation directly from slicer.ini file (this method might be slightly changed in the future)
- to_dict() method of model objects now has a flag create_label which provides label attribute derived from the object’s name, if label is missing
- #95: Allow charset to be specified in Content-Type header
SQL:
- added option to SQL workspace/browser
safe_labels
to use safe column labels for databases that do not support characters like.
in column names even when quoted (advanced feature, does not work with denormalization) - browser accepts include_cell_count and include_summary arguments to optionally disable/enable inclusion of respective results in the aggregation result object
- added implicit ordering by levels to aggregate and dimension values methods (for list of facts it is not yet decided how this should work)
- #97: partially implemented sort_key, available in aggregate() and values() methods
Server:
- added comma separator for
order=
parameter - reflected multiple search backend support in slicer server
Other:
- added vim syntax highlighting goodie
Changes¶
- AggregationResult.cross_table is depreciated, use cross table formatter instead
- load_model() loads and applies translations
- slicer server uses new localization methods (removed localization code from slicer)
- workspace context provides proper list of locales and new key ‘translations’
- added base class Workspace which backends should subclass; backends should use workspace.localized_model(locale)
- create_model() accepts list of translations
Fixes¶
- browser.set_locale() now correctly changes browser’s locale
- #97: Dimension values call cartesians when cutting by a different dimension
- #99: Dimension “template” does not copy hierarchies
0.10.1¶
Quick Summary:
- multiple hierarchies:
- Python:
cut = PointCut("date", [2010,15], hierarchy='ywd')
- Server:
GET /aggregate?cut=date@ywd:2010,15
- Server drilldown:
GET /aggregate?drilldown=date@ywd:week
- Python:
- added experimental result formatters (API might change)
- added experimental pre-aggregations
New Features¶
- added support for multiple hierarchies
- added
dimension_schema
option to star browser – use this when you have all dimensions grouped in a separate schema than fact table - added HierarchyError - used for example when drilling down deeper than possible within that hierarchy
- added result formatters: simple_html_table, simple_data_table, text_table
- added create_formatter(formatter_type, options ...)
- AggregationResult.levels is a new dictionary containing levels that the result was drilled down to. Keys are dimension names, values are levels.
- AggregationResult.table_rows() output has a new variable
is_base
to denote whether the row is base or not in regard to table_rows dimension. - added
create_server(config_path)
to simplify wsgi script - added aggregates: avg, stddev and variance (works only in databases that support those aggregations, such as PostgreSQL)
- added preliminary implemenation of pre-aggregation to sql worskspace:
- create_conformed_rollup()
- create_conformed_rollups()
- create_cube_aggregate()
Server:
- multiple drilldowns can be specified in single argument:
drilldown=date,product
- there can be multiple
cut
arguments that will be appended into single cell - added requests:
GET /cubes
andGET /cube/NAME/dimensions
Changes¶
- Important: Changed string representation of a set cut: now using semicolon ‘;’ as a separator instead of a plus symbol ‘+’
- aggregation browser subclasses should now fill result’s
levels
variable withcoalesced_drilldown()
output for requested drill-down levels. - Moved coalesce_drilldown() from star browser to cubes.browser module to be reusable by other browsers. Method might be renamed in the future.
- if there is only one level (default) in a dimension, it will have same label as the owning dimension
- hierarchy definition errors now raise ModelError instead of generic exception
Fixes¶
- order of joins is preserved
- fixed ordering bug
- fixed bug in generating conditions from range cuts
AggregationResult.table_rows
now works when there is no point cut- get correct reference in
table_rows
– now works when simple denormalized table is used - raise model exception when a table is missing due to missing join
- search in slicer updated for latest changes
- fixed bug that prevented using cells with attributes in aliased joined tables
0.10¶
Quick Summary¶
Dimension defition can have a “template”. For example:
{ "name": "contract_date", "template": "date" }
added table_rows() and cross_table()
added simple_model(cube_name, dimension_names, measures)
- Incompatibilities: use
create_model()
instead ofModel(**dict)
, if you - were using just
load_model()
, you are fine.
New Features¶
To address issue #8 create_model(dict) was added as replacement for Model(dict). Model() from now on will expect correctly constructed model objects.
create_model()
will be able to handle various simplifications and defaults during the construction process.added
info
attribute to all model objects. It can be used to store custom, application or front-end specific informationpreliminary implementation of
cross_table()
(interface might be changed)AggregationResult.table_rows()
- new method that iterates through drill-down rows and returns a tuple with key, label, path, and rest of the fields.dimension in model description can specify another template dimension – all properties from the template will be inherited in the new dimension. All dimension properties specified in the new dimension completely override the template specification
added point_cut_for_dimension
added simple_model(cube_name, dimensions, measures) – creates a single-cube model with flat dimensions from a list of dimension names and measures from a list of measure names. For example:
model = simple_model("contracts", ["year","contractor", "type"], ["amount"])
Slicer Server:
/cell
– return cell details (replaces/details
)
Changes¶
- creation of a model from dictionary through Model(dict) is depreciated, use create_model(dict) instead. All initialization code will be moved there. Depreciation warnings were added. Old functionality retained for the time being. (important)
- Replaced Attribute.full_name() with Attribute.ref()
- Removed Dimension.attribute_reference() as same can be achieved with dim(attr).ref()
- AggregationResult.drilldown renamed to AggregationResults.cells
Planned Changes:
- str(Attribute) will return ref() instead of attribute name as it is more useful
Fixes¶
- order of dimensions is now preserved in the Model
0.9.1¶
Summary: Range cuts, denormalize with slicer tool, cells in /report
query
New Features¶
- cut_from_string(): added parsing of range and set cuts from string; introduced requirement for key format: Keys should now have format “alphanumeric character or underscore” if they are going to be converted to strings (for example when using slicer HTTP server)
- cut_from_dict(): create a cut (of appropriate class) from a dictionary description
- Dimension.attribute(name): get attribute instance from name
- added exceptions: CubesError, ModelInconsistencyError, NoSuchDimensionError, NoSuchAttributeError, ArgumentError, MappingError, WorkspaceError and BrowserError
StarBrowser:
- implemented RangeCut conditions
Slicer Server:
/report
JSON now acceptscell
with full cell description as dictionary, overrides URL parameters
Slicer tool:
denormalize
option for (bulk) denormalization of cubes (see the the slicer documentation for more information)
Changes¶
- all
/report
JSON requests should now have queries wrapped in the keyqueries
. This was originally intended way of use, but was not correctly implemented. A descriptive error message is returned from the server if the keyqueries
is not present. Despite being rather a bug-fix, it is listed here as it requires your attention for possible change of your code. - warn when no backend is specified during slicer context creation
Fixes¶
- Better handling of missing optional packages, also fixes #57 (now works without slqalchemy and without werkzeug as expected)
- see change above about
/report
andqueries
- push more errors as JSON responses to the requestor, instead of just failing with an exception
Version 0.9¶
Important Changes¶
Summary of most important changes that might affect your code:
Slicer: Change all your slicer.ini configuration files to have
[workspace]
section instead of old [db]
or [backend]
. Depreciation
warning is issued, will work if not changed.
Model: Change dimensions
in model
to be an array instead of a
dictionary. Same with cubes
. Old style: "dimensions" = { "date" = ... }
new style: "dimensions" = [ { "name": "date", ... } ]
. Will work if not
changed, just be prepared.
Python: Use Dimension.hierarchy() instead of Dimension.default_hierarchy.
New Features¶
- slicer_context() - new method that holds all relevant information from configuration. can be reused when creating tools that work in connected database environment
- added Hierarchy.all_attributes() and .key_attributes()
- Cell.rollup_dim() - rolls up single dimension to a specified level. this might later replace the Cell.rollup() method
- Cell.drilldown() - drills down the cell
- create_workspace() - new top-level method for creating a workspace by name
of a backend and a configuration dictionary. Easier to create browsers (from
possible browser pool) programmatically. The browser name might be full
module name path or relative to the cubes.backends, for example
sql.browser
for default SQL denormalized browser. - get_backend() - get backend by name
- AggregationBrowser.cell_details(): New method returning values of attributes representing the cell. Preliminary implementation, return value might change.
- AggregationBrowser.cut_details(): New method returning values of attributes representing a single cut. Preliminary implementation, return value might change.
- Dimension.validate() now checks whether there are duplicate attributes
- Cube.validate() now checks whether there are duplicate measures or details
SQL backend:
new StarBrowser implemented:
- StarBrowser supports snowflakes or denormalization (optional)
- for snowflake browsing no write permission is required (does not have to be denormalized)
new DenormalizedMapper for mapping logical model to denormalized view
new SnowflakeMapper for mapping logical model to a snowflake schema
ddl_for_model() - get schema DDL as string for model
join finder and attribute mapper are now just Mapper - class responsible for finding appropriate joins and doing logical-to-physical mappings
coalesce_attribute() - new method for coalescing multiple ways of describing a physical attribute (just attribute or table+schema+attribute)
dimension argument was removed from all methods working with attributes (the dimension is now required attribute property)
added create_denormalized_view() with options: materialize, create_index, keys_only
Slicer:
- slicer ddl - generate schema DDL from model
- slicer test - test configuration and model against database and report list of issues, if any
- Backend options are now in [workspace], removed configurability of custom backend section. Warning are issued when old section names [db] and [backend] are used
- server responds to /details which is a result of AggregationBrowser.cell_details()
Examples:
- added simple Flask based web example - dimension aggregation browser
Changes¶
- in Model: dimension and cube dictionary specification during model initialization is depreciated, list should be used (with explicitly mentioned attribute “name”) – important
- important: Now all attribute references in the model (dimension attributes, measures, ...) are required to be instances of Attribute() and the attribute knows it’s dimension
- removed hierarchy argument from Dimension.all_attributes() and Dimension.key_attributes()
- renamed builder to denormalizer
- Dimension.default_hierarchy is now depreciated in favor of Dimension.hierarchy() which now accepts no arguments or argument None - returning default hierarchy in those two cases
- metadata are now reused for each browser within one workspace - speed improvement.
Fixes¶
- Slicer version should be same version as Cubes: Original intention was to have separate server, therefore it had its own versioning. Now there is no reason for separate version, moreover it can introduce confusion.
- Proper use of database schema in the Mapper
Version 0.8¶
New Features¶
- Started writing StarBrowser - another SQL aggregation browser with different approach (see code/docs)
Slicer Server:
- added configuration option
modules
under[server]
to load additional modules - added ability to specify backend module
- backend configuration is in [backend] by default, for SQL it stays in [db]
- added server config option for default
prettyprint
value (useful for demontration purposes)
Documentation:
- Changed license to MIT + small addition. Please refer to the LICENSE file.
- Updated documentation - added missing parts, made reference more readable, moved class and function reference docs from descriptive part to reference (API) part.
- added backend documentation
- Added “Hello World!” example
Changed Features¶
- removed default SQL backend from the server
- moved worskpace creation into the backend module
Fixes¶
- Fixed create_view to handle not materialized properly (thanks to deytao)
- Slicer tool header now contains #!/usr/bin/env python
Version 0.7.1¶
Added tutorials in tutorials/ with models in tutorials/models/ and data in tutorials/data/:
- Tutorial 1:
- how to build a model programatically
- how to create a model with flat dimensions
- how to aggregate whole cube
- how to drill-down and aggregate through a dimension
- Tutorial 2:
- how to create and use a model file
- mappings
- Tutorial 3:
- how hierarhies work
- drill-down through a hierarchy
- Tutorial 4 (not blogged about it yet):
- how to launch slicer server
New Features¶
- New method: Dimension.attribute_reference: returns full reference to an attribute
- str(cut) will now return constructed string representation of a cut as it can be used by Slicer
Slicer server:
- added /locales to slicer
- added locales key in /model request
- added Access-Control-Allow-Origin for JS/jQuery
Changes¶
- Allow dimensions in cube to be a list, not only a dictionary (internally it is ordered dictionary)
- Allow cubes in model to be a list, not only a dictionary (internally it is ordered dictionary)
Slicer server:
- slicer does not require default cube to be specified: if no cube is in the request then try default from config or get first from model
Fixes¶
- Slicer not serves right localization regardless of what localization was used first after server was launched (changed model localization copy to be deepcopy (as it should be))
- Fixes some remnants that used old Cell.foo based browsing to Browser.foo(cell, ...) only browsing
- fixed model localization issues; once localized, original locale was not available
- Do not try to add locale if not specified. Fixes #11: https://github.com/Stiivi/cubes/issues/11
Version 0.7¶
WARNING: Minor backward API incompatibility - Cuboid renamed to Cell.
Changes¶
- Class ‘Cuboid’ was renamed to more correct ‘Cell’. ‘Cuboid’ is a part of cube with subset of dimensions.
- all APIs with ‘cuboid’ in their name/arguments were renamed to use ‘cell’ instead
- Changed initialization of model classes: Model, Cube, Dimension, Hierarchy, Level to be more “pythony”: instead of using initialization dictionary, each attribute is listed as parameter, rest is handled from variable list of key word arguments
- Improved handling of flat and detail-less dimensions (dimensions represented just by one attribute which is also a key)
Model Initialization Defaults:
- If no levels are specified during initialization, then dimension name is considered flat, with single attribute.
- If no hierarchy is specified and levels are specified, then default hierarchy will be created from order of levels
- If no levels are specified, then one level is created, with name
default
and dimension will be considered flat
Note: This initialization defaults might be moved into a separate utility function/class that will populate incomplete model
New features¶
Slicer server:
- changed to handle multiple cubes within model: you have to specify a cube for /aggregate, /facts,... in form: /cube/<cube_name>/<browser_action>
- reflect change in configuration: removed
view
, addedview_prefix
andview_suffix
, the cube view name will be constructed by concatenatingview prefix
+cube name
+view suffix
- in aggregate drill-down: explicit dimension can be specified with
drilldown=dimension:level
, such as:date:month
This change is considered final and therefore we can mark it is as API version 1.
Version 0.6¶
New features¶
Cubes:
- added ‘details’ to cube - attributes that might contain fact details which are not relevant to aggregation, but might be interesting when displaying facts
- added ordering of facts in aggregation browser
- SQL denormalizer can now add indexes to key columns, if requested
- one detail table can be used more than once in SQL denomralizer (such as an
organisation for both - receiver and donor), added key
``alias``
to``joins``
in model description
Slicer server:
- added
log
a andlog_level
configuration options (under[server]
) - added
format=
parameter to/facts
, acceptsjson
andcsv
- added
fields=
parameter to/facts
- comma separated list of returned fields in CSV - share single sqlalchemy engine within server thread
- limit number of facts returned in JSON (configurable by
json_record_limit
in[server]
section)
Experimental: (might change or be removed, use with caution)
- added cubes searching frontend for separate cubes_search experimenal Sphinx backend (see https://bitbucket.org/Stiivi/cubes-search)
Fixes¶
- fixed localization bug in fact(s) - now uses proper attribute name without locale suffix
- fixed passing of pagination and ordering parameters from server to aggregation browser when requesting facts
- fixed bug when using multiple conditions in SQL aggregator
- make host/port optional separately