DE / EN / IT

tcVISION Extension for BigData

A great part of the added value of modern IT systems is the latency-free data and process integration of transactional and analytical areas. The cross-system integration platform tcVISION is unique, efficient and reliable. With tcVISION mainframe data can be fast and easily integrated into BigData based operative applications or Business Intelligence and Analytics in near real-time.

The tcVISION solution is practice approved and is constantly further developed to meet the requirements of the new technologies. The result is the support of BigData in tcVISION Version 6.

In the current version of tcVISION V6 BigData is a fully integrated output platform and supports the integration with Apache Kafka.

Consequently, tcVISION supports direct streaming of changed data into a BigData Apache Kafka environment. Apache Kafka is an Open Source data streaming platform developed by the Apache Software Foundation.

The software stands out because it is a distributed system and real-time scalable. Thus it is best suited to meet the challenges of BigData requirements.

As with all output platforms provided by tcVISION, the data streaming via Apache Kafka is based on official standard interfaces. The implementation of the Apache Kafka interface is fast and easy.

Data streaming to Apache Kafka as a transport layer enhances the BigData connectivity of tcVISION.

In addition to Apache Kafka, transport layers to BigData include the creation of files, direct output to a hadoop file system (HDFS) as well as the output to MongoDB.

The currently used protocols for data transfer to BigData are JSON, Avro and CSV. The main focus of the tcVISION integration platform is the supply of a real-time synchronization to integrate mainframe data into BigData based solutions.

DB2NachBigData2 en

Features

The tcVISION integration platform consists of a variety of state-of-the-art technology components which cover far more than an ETL process.

  • Data exchange in the sense of real-time synchronization and replication turns into a “Single Step Operation” with tcVISION.

  • No additional middleware is required.

  • Diverse Change Data Capture technologies allow an efficient selection of the required data from the source system with focus on the changed data. The data exchange process is reduced to the necessary minimum which results in lower costs for the data exchange.

  • tcVISION can also use backup and recovery files (e.g. imagecopies, log files, UNLOADs) as a source for replication. Production data does not need to be touched.

  • tcVISION enables the fast and efficient load of large volumes of mainframe data into BigData (streaming). The processor costs of the mainframe are low and negligible.

  • An integrated data repository guarantees transparent data management across platforms.

  • Mainframe knowledge is not necessarily required for the replication.

  • tcVISION includes a rule engine to transform data into a target compliant format or allows user specific processing via supplied APIs.

  • The integrated staging concept supports the offload of changed data in “raw format” to less expensive processor systems. This reduces costs and mainframe processor resources to a minimum.

  • The preparation of the data for the target system can be performed on a less expensive platform (Linux, UNIX or MS-Windows).

  • The transfer to and streaming of data into BigData is part of the tcVISION data exchange process. No intermediate files are required.

  • The exchange of large volumes of data between a production mainframe environment and BigData can run in parallel processes to reduce latencies to a minimum.

  • The tcVISION integration platform contains comprehensive control mechanisms and monitoring functions for an automated data exchange.

  • tcVISION has been designed in a way that BigData based projects can be deployed with complete project autonomy and maximum reduction of mainframe resources.

DB2NachBigData3 en

Benefits

With tcVISION, data synchronization between mainframe and BigData pays off:

  • Near real-time replication of mainframe data to BigData allows actual real-time analytics.

  • The relocation of mainframe applications (e.g. internet applications like online banking, e-Government, etc.) to BigData with synchronous data on both platforms is also possible.

  • Because of the concentration on changed data the costs of the data exchange are reduced to a minimum.

  • The utilization of mainframe resources is reduced as far as possible to avoid costs for mainframe knowedge and mainframe MIPS.

  • Data exchange processes can be deployed and maintained with tcVISION without mainframe knowledge, thus costs can be saved and BigData projects can be developed and put to production faster.

  • The near real-time replication of tcVISION from mainframe to BigData allows the relocation of BI, reporting and analytic applications to the more cost efficient and – for these applications – more powerful BigData platform.

  • Compensation of decreasing mainframe knowledge

tcVISION Integration Platform

The tcVISION Integration Platform is the ideal solution to perform this task:
  • By concentrating on changed data (Changed Data Capture) data exchange processes become very fast and efficient.
  • The transferred data volume is reduced to a minimum.
  • The costs of data transfer from the mainframe to EXASOL are reduced significantly.
  • At the same time the real-time data synchronization capability of tcVISION achieves a very high degree of data timeliness for every EXASOL “Big Data” solution.
  • Mass data exchange can be performed very efficiently.
  • tcVISION supports the parallelization of data transfers from the mainframe to EXASOL “Big Data”. This reduces the latency times to a minimum.
  • All data exchange processes are documented and comprehensible.
  • Today mainframe resources are limited and expensive. Using the tcVISION solution the mainframe resources required for the integration are reduced to a minimum (manpower, processor costs). Analytics and BI solutions with total project autonomy can be deployed across system boundaries.

In-Memory Technology

Data and informations have become an important factor in production and everyone is talking about "big data". But what is big data? According to Gartner Glossary the following is characteristic for big data:

  1. constantly growing amounts of data
  2. constantly growing speed with which data are created and processed
  3. a growing diversity of data

The new data diversity does not only include structured but also unstructured data from the internet, e.g. images, sound files and videos. This is a problem for traditional methods of data analysis.

  • In-Memory Technology

In-Memory data bases open new possibilities to enterprises that use mainframes. Large amounts of data (mainframe, open systems, social media, internet, internet of things) can be analyzed together. This helps to recognize trends, find new sources of earnings, develop products, optimize processes, make prognoses, calculate risks and save costs. Data and their analysis are the keys to the success of an enterprise.

  • The Task

With respect to big data, the mainframe and the data stored on it play a role that may not be underestimated. It is believed that still over 80% of transactional data resides on mainframes. The history of the mainframe is over 50 years long. Its data structures have grown historically, are often complex and not compatible to big data. Including the mainframe in big data is a challenge for most enterprises. Integrating the mainframe not only means extracting the data from mainframe DBMS but also converting it to a big data compatible format.

  • Mainframe Integration in Real-Time

The combination of in-memory technology and tcVISION revolutionizes customers' possibilities of big data analyses of all kinds. tcVISION makes it possible to integrate mainframe data into in-memory databases in real-time. Mainframe data can be included in a real-time analysis at any time

tcVISION can integrate mainframe data from Db2, VSAM, IMS/DB, DL/I, ADABAS, CA-DATACOM/DB and CA-IDMS/DB with all their special features into in-memory databases of different manufacturers. - in real-time.

This site uses cookies.

By continuing the use of this site, you agree to it. Read our privacy statement here: Learn more

I understand