What's new in the v2.4 stable release series

v2.4.3.0 - May 12, 2021

Downloads

macOS       Linux

Docker

Improvements

Core database

• [8052] Add ability to configure cipher list

Yugabyte Platform

• [1342] Fixing the error message when the get host info call to gcp fails

• [8165]Yugaware creates zombie ssh process

Bug fixes

• [8176] GCP universes don't have correct nproc system settings.

Core database

• [8150] [8196] Fix preceding op id in case of empty ops sent to the follower

• [8348] Correctly handling a failure to create a priority thread pool worker thread

Yugabyte Platform

• [6758] Unable to reuse On-premise instance incase previouse universe failed / deleted (#8263)

v2.4.2.0 - May 3, 2021

Downloads

macOS       Linux

Docker

Improvements

Core database

• [5647] docdb: LB should not move tablets right after splitting
• [6241] YSQL: Make row key share lock less restrictive
• [6305] Adaptive Heartbeat Reporting
• [6445] docdb: Master should rebuild YCQL system.partitions on a background thread
• [7079] [7153] YSQL: Improve configuration and defaults for client to server encryption
• [7355] ysql: check backfill bad connection status
• [7455] [YCQL] Update index from transaction with cross-key statements.
• [7641] YCQL: Fix checks in index update path that determine full row removal.
• [7705] ysql: prioritize internal HBA config
• [7813] [YSQL] YSQL dump should always include HASH/ASC/DESC modifier for indexes/pkey.

Yugabyte Platform

• [6668] Support platform backup/restore on Kubernetes

Bug fixes

Core database

• [5383] Fix conflict resolution while adding read intents
• [6747] [6902] [7109] [7284] YCQL system.partition bug fixes
• [6960] docdb - Register ScopedRWOperation when accessing doc_db data in StillHasParentDataAfterSplit
• [7105] Explicitly clear the global PgMemctx map in YBCDestroyPgGate
• [7145] Fix wrong propagated safe time
• [7397] Tablet splitting: Crash during generating TSHeartbeat report after split
• [7835] Don't crash when trying to append ValueType::kTombstone to a key

Yugabyte Platform

• [1342] [Platform] Fixing the error message when the get host info call to cloud providers fails
• [6698] [7262] Platform: Cascade delete alert and alert_definition
• [7054] [Platform] Add conditional checks for hiding specific platform elements in non-platform mode.
• [7268] [Platform] Use the correct SSH user for on-prem node preflight checks
• [7385] [Platform] Missing package for s3cmd dependency for airgapped GCP VMs
• [7390] [Platform] Preflight checks should handle sudo passwords when given
• [7402] [Platform] parsing of df output is fragile and may fail in case of "safe" error in df
• [7408] [Platform] Insufficient user feedback in platform for Task Retry
• [7416] [Platform] Changed default port of On-Prem provider to 22 (#7599)
• [7475] [Platform] Use more recent CentOS-7 base image for GCP universe VMs
• [7597] [Platform] Add python modules to PYTHONPATH in non-replicated installations
• [7656] [Platform] Use the correct SSH user + port for onprem workflow.
• [7698] [Platform] Custom SMTP Configuration API returns unmasked SMTP password
• [7703] [Platform] Can't send email for custom SMTP settings without authentication (empty username)
• [7704] [Platform] Backup to S3 fails using Platform instance's IAM role
• [7769] Prevent adding on-prem node instance with duplicate IP
• [7771] [Platform] A lot of exceptions in logs "Error querying for alerts..."
• [7780] [Platform] Make backup deletion independent of schedule
• [7810] [Platform] Update SMTP configuration for health check emails
• [7909] [Platform] Fix issue with signature could not be verified for google-cloud-sdk for GCP VMs
• [7918] [Platform] Add US-West2 GCP metadata to Platform
• [7988] [Platform] Backup deletion script shouldn't cause retries

v2.4.1.1 - March 30, 2021

Downloads

macOS       Linux

Docker

Bug Fixes

Core Database

• [7145] Fix wrong propagated safe time

Yugabyte Platform

• [7385] Platform: Missing package for s3cmd dependency for airgapped GCP VMs

v2.4.1 - Feb 10, 2021

Downloads

macOS       Linux

Docker

Improvements

Yugabyte Platform

• When creating/restoring backups, use the local UNIX socket to connect to YSQL instead of the hostname if authentication is enabled [5571]
• On universe creation, platform checks that clocks are synchronized [6017]
• Health Check generates alerts if it isn't running [6581]
• Manual provisioning now runs the same preflight checks as regular provisioning [6819]
• Retrying a failed task now provides visual feedback [6820]
• Removed the "Transactions" plot line from YSQL Ops and Latency graphs [6839]
• You can now specify a minimum TLS version for Platform [6893], [7140]
• Health checks now default to TLS 1.2 [7196]

Core Database

• The YB-TServer metrics output (for example, on localhost) now shows the total affected rows for each operation. [4600]
• YSQL: Backup for colocated databases [4874]
• Optimizations to the YSQL layer to apply empty deletes only when required [5686]
  • Backup: Fix restore of colocated table with table_oid already set [6678]
• Metrics thread now start after the first replication stream is created [5251]
• The YB-TServer metrics output (for example, on localhost) now shows transaction BEGIN, COMMIT, and ROLLBACK statements. [6486]
• Restore now preserves the exact partitioning of the source tablets [6628]
• Enabled a sanity check to ensure that the tablet lookup result matches the partition key [7016]

Bug Fixes

Yugabyte Platform

• Release instance was not an option for a node if an install failed because of SSH access [5942]
• Backup-related tasks (schedules, restores, deletes) failed when the storage configuration was deleted [6680]
• VPC cross-linking failed during creation of an AWS provider [6748]
• Fixes to YSQL backups with node-to-node TLS encryption enabled [6965]
  • Add certs flags to ysql_dump when backing up a node-to-node TLS-enabled universe
• Corrected an error when backing up multiple YSQL namespaces in a universe that is encrypted at rest [7114]
• Fixed a syntax error in replicated.yml [7180]
• Fixed an issue preventing health checks from using an appropriate TLS version [7196]

Core Database

• YSQL: Fix calculation for transaction counts in YSQL metrics. [4599]
• ybase: a blacklisted TS' initial load was not replicated during master failover [6397]
• The CREATE INDEX statement generated by ysql_dump now uses SPLIT INTO syntax [6537]
• YCQL: Fixed a bug with an index update on a list after an overwrite [6735]
• Fixed a bug in YSQL for online index backfill capabilities, to ensure deletes to an index are persisted during backfill [6811]
• YCQL: Fixed various issues for literals of collection datatypes [6829], 6879
• The permissions map is now rebuilt after all ALTER ROLE operations [7008]
• Fixes to Async Replication 2DC capabilities, including a bug fix for a race condition on consumer with smaller batch sizes [7040]

v2.4.0 - Jan 22, 2021

Downloads

macOS       Linux

Docker

New Features

Yugabyte Platform

• Data encryption in-transit enhancements:

  • Support for bringing your own CA certificates for an on-premise cloud provider (5729)
  • Support for rotating custom CA certificates (5727)
  • Displaying the certificate name in the YB Platform Universe editor
  • Moving of the Certificates page to Cloud Config > Security > Encryption in Transit

• Moving of TLS In-Transit Certificates to Cloud > Security Config

• Support for Rolling restart of a universe (6323)

• Support for VMware Tanzu as a new cloud provider (6633)

• Alerts for backup tasks (5556)

Core Database

• Introducing support for audit logging in YCQL and YSQL API (1331, 5887, 6199)
• Ability to log slow running queries in YSQL (4817)
• Introducing support for LDAP integration in YSQL API (6088)

Improvements

Yugabyte Platform

• Support for Transactional tables and Cross Cluster Async Replication topology (5779)

• Support for very large transactions and stability improvements (1923)

• Displaying an entire query in the detailed view of the live queries tab (6412)

• Not returning Hashes and Tokens in API responses (6388)

• Authentication of proxy requests against a valid platform session (6544)

• Disabling of the unused API endpoint run_query (6383)

• Improved error handling for user-created concurrent backup tasks (5888)

• Performance improvements for live queries (6289)

• Pre-flight checks for Create Universe, Edit Universe, and Add Node operations for an on-premise provider (6016)

• Disabling of the unused API endpoint run_in_shell (6384)

• Disabling of the unused API endpoint create_db_credentials (6385)

• Input validation for Kubernetes configuration path (6389)

• Input validation for the access_keys API endpoint (6386)

• Input path validation for backup target paths (6382)

• Timeout support for busybox (6652)

• Enabling of CORS policy by default (6390)

• Deleting backups for TLS-enabled universes (5980)

Core Database

• DDL consistency (4710, 3979, 4360)

• DDL performance improvements (5177, 3503)

• ALTER versions for ORM support (4424)

• Online index backfill GA (2301, [4899], 5031)

• Table partitioning (1126,5387)

• SQL support improvements:

  • Support for the FROM clause in UPDATE (738)
  • Support for the USING clause in DELETE (738)
  • SQL/JSON path language [PG12] (5408)

• Performance improvements:

  • Sequence cache min-value flag (6041, 5869)
  • Foreign-key batching (2951)

• YSQL usability improvements:

  • COPY FROM rows-per-txn option (2855, 6380)
  • COPY / ysql_dump OOM (5205, 5453, 5603)
  • Large DML OOM (5374)
  • Updating pkey values (659)
  • Restarting write-txns on conflict (4291)

• YSQL statement execution statistics (5478)

• Improved raft leader stepdown operations, by waiting for the target follower to catch up. This reduces the unavailability window during cluster operations. (5570)

• Improved performance during cluster overload:

  • Improved throttling for user-level YCQL RPC (4973)
  • Lower YCQL live RPC memory usage (5057)
  • Improved throttling for internal master level RPCs (5434)

• Improved performance for YCQL and many connections:

  • Cache for generic YCQL system queries (5043)
  • Cache for auth information for YCQL (6010)
  • Speedup system.partitions queries (6394)

• Improved DNS handling:

  • Process-wide cache for DNS queries (5201)
  • Avoid duplicate DNS requests during YCQL system.partitions queries (5225)

• Improved master-level load balancer (LB) operations:

  • Throttle tablet moves per table (4053)
  • New global optimization for tablet balancing -- this also allows the LB to work with colocation (3079)
  • New global optimization for leader balancing (5021)
  • Delay LB on master leader failover (5221)
  • Register temporary TS on master failover to prevent invalid LB operations (4691)
  • Configurable option to disallow TLS v1.0 and v1.1 protocols (6865)

• Skip loading deleted table metadata into master memory (5122)

Bug Fixes

Yugabyte Platform

• Edit Universe did not display the TLS certificate used
• Multi-zone K8s universe creation failed (5882)
• Tasks page reported an incorrect status of a failed backup (6210)
• K8s universe upgrade did not wait for each pod to update before moving to the next (6360)
• Changing node count by AZ was doing a full move (5335)
• Enabling Encryption-at-Rest without KMS configuration caused create Universe to fail silently (6228)
• Missing SST size in the Nodes page of a universe with read replicas (6275)
• Incorrect link in the Live queries dashboard node name (6458)
• Failure to create a TLS-enabled universe with a custom home directory setting with on-premise provider (6602)
• Database node health liveliness check was blocked indefinitely (6301)

Core Database

• Critical fixes for transaction cleanup applicable to aborted transactions (observed frequently as servers reaching soft memory limit)

• Main raft fixes:

  • Incorrect tracking of flushed op id in case of aborted operations (4150)
  • Various fixes and potential crashes on tablet bootstrap (5003, 3759, 4983, 5215, 5224)
  • Avoid running out of threads due to FailureDetector (5752)

• Metrics reporting could be inconsistent due to regular and intent rocksdb using the same statistics object (5640)

• Various issues with the rocksdb snapshot mechanism used for backup/restore (6170, 4756, 5337)

• Fixes in the YCQL API:

  • Fix parser typo when scanning binary values (6827)
  • Omit where clause if there is no range column (6826)
  • Fix JSONB operator execution when applying to NULL objects (6766)
  • Fix range deletes to start and end at specified primary key bounds (6649)

• Fixes in the YSQL API:

  • Fix crash for large DDL transaction (6430)
  • Fixed SIGSERV in YBPreloadRelCache (6317)
  • Fix backup-restore issues when source table was altered before (6009, 6245, 5958)
  • Fix backup-restore failure due to unexpected NULLABLE column attribute. (6886)

• Retry on SSL_ERROR_WANT_WRITE (6266)

Known Issues

Yugabyte Platform

• Azure IaaS orchestration (in beta status):

  • No pricing information (5624)
  • No support for regions with zero availability zones (AZs) (5628)

Core Database

• Automatic Tablet Splitting:

  • While a tablet split is occurring, in-flight operations for both YCQL and YSQL APIs would currently receive errors. These would currently have to be retried at the application level currently. In the future, these will be transparently handled underneath the hood. The immediate impact for this would be that certain tools like TPCC or sysbench would fail while tablet splitting is happening.

Notes