Needed: Automated Database Integration

 It’s long been a truism that data is one of the key assets, if not the key asset, for an enterprise. Our recent conversations with customers are putting a new spin on this insight. They are telling us why database integration and automation are increasingly important to them to not only reduce costs, but to increase agility. Integration Requirements By integration we mean the ability to quickly and easily combine data from multiple source relational databases into one, or more, central databases while keeping the original databases intact. Such integration allows customers and suppliers to share production and parts information without the expense of implementing EDI. Field auditors at a federal agency are using our own DataPortal to easily share relational databases of audit records to give managers a cross-geography view of agency performance. An online retailer could use such integration to combine information from multiple suppliers and distributors into a single database so web customers can get an up-to-date view of stock on hand and ship dates. Much of this information is stored in relational databases. Our customers are telling us they need to integrate these databases without stripping out the table structures and table references that allow ad-hoc queries. They also need to retain the complex data types (anything from engineering change orders to marketing images) they store in relational platforms. Customers also need to be able to identify which database fields originated in which source database. Such tracking by the source of information is important for anything from identifying which supplier has the lowest prices or most inventory to identifying the source of cost or sales issues within multiple business units. Customers also tell us they need to perform this integration easily and quickly. Ideally, they’d like to see it automated. This not only reduces data management costs, but helps an organization respond with greater agility to changing business needs. For example, the more quickly a major aerospace manufacturer can see database information from new suppliers, the more quickly it can fill its needs for critical parts. The more quickly a CFO can see integrated cost and sales databases from various business units, the more quickly he can see and fix problems.  Our Solution Traditionally, database integration has required days, weeks or months of work by skilled database administrators using database management tools. These days, that is unfeasible both from a cost and a time perspective....

Read More

Getting Big Data Where It Needs To Go

Getting Big Data Where It Needs To Go

Information is a critical, if not the, critical business asset in a fast-changing world. The more information an organization has, the better it can make decisions about which products and services to offer, how to price and distribute them, where it can reduce costs and where it can find new customers. Big Data is the process of creating insight from the increasing masses of raw data every organization generates.  Most Big Data is made up of loosely structured or unstructured data such as text, photos, genomic or other scientific data. But some of it also originates in structured relational databases. In addition, many of the results of a Big Data analysis are distributed in the form of a relational database. Whenever Big Data relies on relational data as either a source of information or a way to distribute Big Data insights, DataPortal can share those relational databases quickly, easily, cost-effectively and securely. “Point and Click” Database Sharing DataPortal is a patented tool for moving, storing, and sharing database data across platforms and over the Web. It was designed to be used by people at all technical levels. It requires no coding, entering command lines, opening ports or installation of software. Using DataPortal, each database is published to a DataPortal server. From there it is available over the internal LAN or the Internet to any authorized DataPortal client. This server-to-client communication avoids the security risks of exposing the database directly to a WAN or the Internet. Any user can transfer a full relational database or a subset of that database, in relational form, with the push of a button. All they need is a Web browser and authority to share the database. Using DataPortal, subject matter experts can stay focused on their critical, value-added analytic work without wasting time on the mechanics of sending and receiving data. DataPortal for Data Gathering DataPortal is an efficient way to gather structured data from various database sources at a central location for Big Data integration, analysis, conversion and storage. DataPortal complements Big Data frameworks such as Hadoop, which allow users to access massive quantities of data spread over hundreds or even thousands of processors. These frameworks allow queries to be performed using the familiar SQL used in relational databases.  The results of such queries can also take the form of structured data. Sharing the result set returned by Hadoop with end users can be...

Read More

The Agility Tax: The Hidden Price of EDI

The Agility Tax: The Hidden Price of EDI

If you’re a large manufacturer and need access to a rich Web of suppliers — or a supplier looking to sell to a major manufacturer — you know the critical importance of keeping your costs down. But you also know that one cost that can damage your business the most is not being able to share databases quickly enough to take advantage of a new opportunity. This agility tax kicks in when, say, an auto parts manufacturer finds itself short of a specialty heat-treated connector due to a burst in sales or an earthquake near their overseas supplier. If its IT systems can’t easily share the databases of production forecasts and parts it needs with potential new suppliers, production will stop and it will lose sales to its competitors. The same is obviously true for potential new suppliers, who may be able to beat the incumbent on price or quality, but can’t bid for the business because it would take too long to share the required databases. EDI: Right Answer, Wrong Problem Often, the legacy system standing in the way of new business opportunities like these is EDI (electronic data interchange.) Most major manufacturers, suppliers, shippers and retailers (not to mention government agencies) have used EDI for decades to eliminate the use of paper in common business transactions. Despite its name, EDI was designed not to share data in the form of relational databases, but to share standard business forms such as purchase orders, invoices, shipping notices and carrier-to-carrier waybills. It does a good enough job of solving that problem, assuming you and your business partner have enough volume to justify its expense and complexity. But if you need to add new business partners quickly – say, in the wake of a strike, natural or man-made disaster that takes a key supplier offline – EDI is not a good way to quickly share relational databases containing the production and order data you need to ramp up quickly. To know why, it helps to understand what it takes to implement EDI. EDI Under the Covers EDI is a software interface that sits between the different computer systems, and the different data structures, used by the sender and receiver’s systems. That means both parties must buy, configure and maintain EDI software to convert the sender’s form or data into a standard EDI transaction, and to convert the form into a format...

Read More

What You Don’t Know About Database Sharing Could Hurt You

What You Don’t Know About Database Sharing Could Hurt You

If you’re not part of the corporate IT staff, should you care about the nitty-gritty of how your organization shares complex relational databases? The answer is yes, if you are: A Fortune 2000 organization that could better fill rush orders if it could bring new suppliers on board more quickly. A small, specialty supplier that could sell to more, and bigger, customers if it could easily access their production forecasts. A growing business in an emerging market that needs to avoid the complexity and expense of electronic data interchange just to share business data. A CFO who could understand his business better if his financial analysts could more easily access and drill into raw financial data. A Big Data service provider that could make more sales if it could more easily share its insights and data results with its customers, or Any organization that could cut costs or increase sales by doing a better job analyzing data. Why “Relational” Matters If you’re storing any substantial amount of information about customers, distributors, suppliers, raw materials, products, sales, costs or profits, it’s probably in a relational database.  This “relational” capability is critical because it allows you to explore the relationships among different subsets of the data in unpredictable ways to meet new challenges. Let’s say, for example, you’re a sales manager for a retail chain trying to understand why sales fell so far in January. You suspect it’s due to a flu outbreak, and knowing the answer could help you better track future flu outbreaks and adjust stock and promotions accordingly. You have your business analysts compare county-by-county sales of specific products with state data on county-by-county instances of the flu. This is the type of ad-hoc query that’s so important to the business, and is the core function of a relational database.  Even just sharing production forecasts between a customer and supplier require a relational format so the customer can compare suppliers on metrics such as price, on-time delivery and quality. Yet it is exactly these business-critical relational capabilities that most current database sharing approaches strip away – or preserve only at a cost that is too high for most customers. Key Requirements Retaining table structures is important because database queries require them. Comparing specific product sales per store against county-level flu levels might require one table listing sales per store, another listing flu cases per county, and yet another table...

Read More