Revisions

January 3, 2024

• Added two open ports to the image

  Sometimes there is a requirement to add another service to the watsonx.data image. For instance, you may want to add MongoDB or MSSQL to the system in order to demonstrate federations with these data source. Since we do not know what your requirements are, we have opened up two ports that can be assigned to any service. The documentation has been updated to describe what steps are needed to use these open up and use these ports.

December 6, 2023

• Updated the lab to GA 1.1.0 code

  • What's new in watsonx.data version 1.1.0 Reference

  • Time-travel and roll-back queries

    You can now run the following time-travel queries to access historical data in Apache Iceberg tables:

    SELECT <columns> FROM <iceberg-table> FOR TIMESTAMP AS OF TIMESTAMP <timestamp>
    SELECT <columns> FROM <iceberg-table> FOR VERSION AS OF <snapshotId>
    

    You can use time-travel queries to query and restore data that was updated or deleted in the past.

    You can also roll back an Apache Iceberg table to any existing snapshot.

  • Capture historical data about Presto queries

    The Query History Monitoring and Management (QHMM) service captures historical data about Presto queries and events. The historical data is stored in a MinIO bucket and you can use the data to understand the queries that were run and to debug the Presto engine.

  • Improved query performance with Metastore, File list, and File metadata caching

  • You can now capture and track the DDL changes in watsonx.data by using an event listener.

  • Ingest data by using Spark

    You can now use the IBM Analytics Engine powered by Apache Spark to run ingestion jobs in watsonx.data.

  • Integration with Db2 and Netezza Performance Server

    You can now register Db2 or Netezza Performance Server engines in watsonx.data console.

  • New connectors

    You can now use connectors in watsonx.data to establish connections to the following types of databases:

    • Teradata
    • Delta Lake
    • Elasticsearch
    • SAP HANA
    • SingleStoreDB
    • Snowflake
    • Teradata

• Db2 Upgraded to 11.5.9

  • What's new in Db2 11.5.9 Reference

October 6, 2023

• Updated the lab to GA 1.0.3 code

  • What's new in watsonx.data version 1.1.0

• Image now available in 10 data centers with simpler provisioning

  • Requesting an Image

• Removed VPN Requirement

• External URLs and Ports for all UI Services

  • watsonx.data Ports

• Added PostgreSQL and MySQL databases

  • Postgres Connection
  • MySQL Connection

• Added Jupyter notebook examples

  • Jupyter Notebook support

• Fixed Presto certificate to support TechZone addresses without updating/etc/hosts

  • Watsonx.data Connection Certificate

• Added standalone Spark server to show connectivity to the Presto database

  • Accessing watsonx.data with Spark

• Added watsonx.data Client code

  • Watsonx.data client utilities

• Added MinIO CLI interface

  • MinIO CLI

• Exposed external ports for MinIO, Db2, MySQL, PostgreSQL, Hive, PrestoDB

  • watsonx.data Ports

• VNC Interface disabled by default

  • Enabling VNC Access

• Added Ingesting data chapter

  • Ingesting Data

July 25, 2023

• Updated the lab to GA 1.0.1 code
• Automated start of watsonx.data and simplification of many of the sections
• Removed the Ingest section until a new version is available
• Added Db2 and PostgreSQL connection details

June 12, 2023

Clarified some commands and added an Appendix on common issues.

June 6, 2023

Updated instructions for new TechZone image and added Ingest lab instructions.

May 25th, 2023

Initial publication.