In Teiid Designer, we can expose Virtual Databases (VDBs) as RESTful services by generating a REST WAR containing REST web service procedures. We have enhanced that capability in the 9.1 release (in beta as of this writing) by adding Swagger annotations that expose the service metadata and allow you to view and execute the procedures in the generated WAR. Swagger is an open-source framework that allows you to document your REST APIs.


Here is a video that illustrates the feature:

IMG_2487.JPG copy.jpg

First Impressions

 

I have attended multiple SAP techEd and d-code conferences over the years, but this was my first Sapphire. As anticipated, it was a different perspective in terms of audience. At techEd and d-code, most are developers with an in-depth knowledge SAP ABAP, development, data integration, open standards and other technical topics. At Sapphire, the average attendee was more business-minded, but many came from a technical background. There were many architects and product managers for example.

 

My previous webinars and talks on SAP integration with Red Hat JBoss Middleware were all heavily slanted towards the technical details. This time around I had talks discussing our Data Virtualization integration with SAP HANA. The fact that my session was devoid of the technical overtones from previous presentations, gave me the chance to appeal to the business case for our Data Virtualization solution for SAP HANA. I was able to take the time to discuss the issues with enterprise data management, the weaknesses of existing integration technologies, and where Red Hat JBoss Data Virtualization fills the gaps. I then followed the discussion with a drag-and-drop demo that federated data from SAP HANA and a MySql instance in Teiid Designer, the tooling for Data Virtualization. The talk and demo was well-received. I think explaining enterprise data needs, why current solutions fall short and how data virtualization can do what no other technology can to fulfill real-time data federation requirements was a huge eye-opener. Some expressed how they had no idea this technology was even available and the many man-hours it will save. This conference has given me a new perspective. I may re-purpose this presentation for a future, more technical conference. Developers will certainly appreciate the business value as well as the technical value of Data Virtualization.

 

sapphire_talk.png

Takeways

 

Every time I attend a conference and pull booth-duty, I learn so much from my colleagues regarding other Red Hat products. This time I got a full serving of GSS (Global Support Services) knowledge and fully understand why we have the best customer support organization in the industry. We never rest on our laurels and are always looking at ways to improve and grow. Not only are we the most compelling and cost-effective technical option with our full stack of solutions, we ice the cake with our best-in-class services and support.

 

Not to be lost in all my Red Hat cheerleading (sorry.. just calling it like I see it), is the fact that our partnership with SAP is stronger than it has ever been. I have been working with my SAP colleagues for so long, it feels like we are one big company. The value of an SAP plus Red Hat integration strategy in the enterprise is as compelling as it is due to a level of cooperation and synergy seldom seen in technology today. The benefactors are our customers whom are reaping the rewards through cost-savings, efficiencies, and productivity never realized before.

 

Please feel free to reach out to me if you have any questions about our SAP integration solutions with Red Hat JBoss Middleware and follow me on twitter @jonested. I look forward to SAP d-code in November where we have big plans for unveiling more collaboration between SAP and Red Hat. Hope to see you there!

 

I attended SAP d-code Las Vegas and Berlin representing Red Hat at the Open Source bar of the Hacker's Lounge. The Open Source bar is a new concept for SAP and shows how far they have come in embracing and seeing the value of open source. In addition to Red Hat, GitHub, CloudFoundry, OpenStack, and SUSE participated in the bar.

 

B2UFrjIIUAAGiwf.jpg

 

Although we had a Red Hat booth (which was well-attended), the intention of the Open Source bar was to encourage conversation between OSS companies and SAP developers. There were many whom were not familiar with OSS or knew about it but did not realize the enterprise-class value it offered. We featured a demo using data virtualization running in OpenShift and consuming data from an SAP SQL Anywhere OpenShift instance and an external SAP Gateway instance. The unified view was then consumed by a native Android application.

 

We also had a couple of talks at both events that covered mobile development in the cloud using open-source technologies (PaaS, Data Virtualization, Android). The talks were recorded:

Las Vegas Tech Talk

Berlin Tech Talk


The beer and light up pilsner glasses were a hit in Las Vegas while the red Fedoras and challenging puzzle in the Berlin Red Hat booth kept traffic flowing. There were many great conversations on RHEL with SAP, HANA, OpenShift, Data Virtualization and many other hot topics. Although I seldom (read as never) get a chance to attend any of the sessions due to booth work, talks, etc., SAP TechEd/d-code is always one of my favorite conferences. They do everything first class and make it a valuable experience for all in attendance. I hope to go to the Shanghai d-code in March to spread some OSS love there as well!


Red Hat JBoss Data Virtualization allows you to expose multiple data sources as a single virtual database. Imagine pulling in all of your various sources together into a single, updatable view or series of views using a single point of connection. Now imagine doing that in the cloud using OpenShift! This article will walk through the steps to accomplish just that. The only prerequisite is that you have an SAP Gateway instance available. The demo uses the SAP Gateway demo system described here: http://scn.sap.com/docs/DOC-40986.

 

Here are the steps:

 

1. Setting Up a Data Virtualization instance on OpenShift.

2. Adding a SQL Anywhere instance in OpenShift.

3. Importing from SQL Anywhere and SAP Gateway.

4. Federating those SAP source into a single virtualized view that will be consumed by an Android application in JBoss Developer Studio (JBDS).

5. Run the Android application.

 

Here is a diagram that illustrates the architecture of this example:

sap gateway and sql anywhere dv architecture.png

 

1. Setting up Data Virtualization in OpenShift

 

The first step will be to provision Data Virtualization on OpenShift and install JBDS with Teiid Designer, the design tool for Data Virtualization. See this article for details: Provision Data Virtualization on OpenShift and Connect from Teiid Designer.

 

We will be working with an Android application, so you will need to install the Android plugin into JBDS. See http://developer.android.com/sdk/installing/installing-adt.html. Also, you will need to clone the Android application and artifacts from tejones/jboss-sap-integration · GitHub.

 

2. Adding a SQL Anywhere Instance in OpenShift

 

Now that we have our tooling and our OpenShift Data Virtualization instance running, we are now ready to create our SQL Anywhere database on OpenShift.

From a command prompt, type:

 

# rhc cartridge add https://github.com/githubuser8/openshift-origin-cartridge-sybase-sa/raw/master/metadata/manifest.yml -a <your app>

























 

After adding the cartridge, you will have to accept the license agreement. There are instructions generated by the installation. After starting the ASA instance, refresh the OpenShift instance in JBDS to pickup the ASA and you can begin port forwarding in JBDS.

 

We will need to add our data the SQL Anywhere instance. Any JDBC capable SQL tool will suffice, but I use and love SQuirreL http://squirrel-sql.sourceforge.net/.

The DDL is located in {local_git_repo_location}/jboss-sap-integration/android-dv-sap/demo resources/ddl/flight_sqlana.ddl. I've also included a SQL Anywhere driver for use in the demo located here: {local_git_repo_location}/jboss-sap-integration/android-dv-sap/demo resources/driver/jconn4-26502.jar. You will need that to create the driver setup and connection in SQuirreL. You will use your local host IP in the connection URL since you are using port forwarding. The default credentials are username=dba and password=sql. Here is what your connection in SQuirreL should look like:

 

squirrel_asa_connection.png

You are now ready to connect to the SQL Anywhere source in OpenShift and execute the DDL to create your objects and source your data. Note that the demo data is specific to the data coming from the SAP Gateway demo system. You may need to modify the information in the liveFlightFeed to match up with the test data coming from the Gateway FlightCollection. I am targeting Nov 26, 2014 for this demo. You can change the data to whatever you want. The liveFlightFeed is really meant to simulate a real-time flight data source such as FlightStats.

 

3. Importing from SQL Anywhere and SAP Gateway

 

As I have stated, I will connect to the SAP Gateway Demo system and use the Flight data for this example, (https://sapes1.sapdevcenter.com/sap/opu/odata/IWFND/RMTSAMPLEFLIGHT/). To access this data, you need an account. To create relational objects that map to the collections and functions in the SAP Gateway Service, we need to create a data source for the service and then import into Teiid Designer using the Teiid Connection import option. We will do the same for the SQL Anywhere instance so let's begin there.

 

1. Launch JBoss Developer Studio and switch to the Teiid Designer perspective.  The perspective options are shown in the upper right corner of JBDS.

2. Connect to the Data Virtualization instance in Teiid Designer as demonstrated in step 1.

3. Create a Model Project (or you can use any existing project).

  • In ModelExplorer, Right-click > New > Teiid Model Project
  • In the Model Project wizard, enter Demo for the Project name.  Click Finish to create the empty project.

4. Select the project, then Right-click > Import... You will see the import dialog (shown below).  Select Teiid Connection >> Source Model then click Next.  We are using the Teiid Connection Importer since we cannot directly connect to the DB port on OpenShift.  This importer allows us retrieve the DB schema through Teiid.

5. On the first page of the import wizard, click the New button in the DataSource section.  Click Add Driver to select the jconn4-26502.jar file from the driver directory of your local git project. Name the datasource Flight_Sybase_ASA and enter jdbc:sybase:Tds:{IP for OpenShift ASA instance}:2638/demo?ServiceName=demo. The username and password are dba and sql. Click Apply and then click OK. Click Next on the importer wizard.

createDatasource.png

6. On the next page of the wizard, set the Translator type to Sybase and add an optional source import property with the name importer.tableTypes and the value TABLE. This will avoid importing system table for the SQL Anywhere instance. Click Next.

teiidConnImportPage2.png

7. On the next page, select a project and give the model name myflight. Click Next. This will deploy a temporary dynamic VDB that will enable the metadata mapping from SQL Anywhere objects the Data Virtualization relational objects by creating DDL.

teiidConnImportPage3.png

8. You will now see the DDL that will be used to create the Data Virtualization relational object, click Next to see the objects that will be created and then click Finish.

teiidConnImportPage4.png

This will create the objects for us in the ModelExplorer.

ASA_Teiid_Objects.png


Creating the SAP Gateway Relational Objects

The process for importing the SAP Gateway source is a lot like importing SQL Anywhere except the connection information is different and we will use a different translator.


1. Repeat step 4 from above. When creating the datasource for SAP Gateway as in step 5 above, name the datasource SAPGatewayFlight and select the webservice driver. Enter https://sapes1.sapdevcenter.com/sap/opu/odata/IWFND/RMTSAMPLEFLIGHT for the URL, End Point and HTTPBasic for security. The username and password are your credentials for the SAP Gateway service you are using. Click Apply and then click OK. Click Next on the importer wizard.

createGatewaySource.png

2. On the next page, select the sap-nw-gateway translator and click Next. Enter SAPGatewayFlight for the model name and click Next.

3. Complete step 8 above and the relational objects that map to the SAP Gateway service will be generated for you in JBDS.

Note: With the current version of the Data Virtualization cartridge for OpenShift, there is an error in the mapping of the "byte" data type. The flightDetails_period field of the FlightCollection table should be change to "short". This will be corrected in the next release of the cartridge. To change the type, select the field and click on the ellipsis of the "datatype" property in the properties panel below. A data type dialog will popup allowing you to select the "short" type.

Gateway_Teiid_Objects.png


4. Federating SAP Sources into a Single Virtualized View

 

Now that you have imported the SQL Anywhere source from OpenShift and the SAP Gateway OData service, you are ready to join the data into a single view. To start, we will need to create a virtual model and table that will be use to federate the data from our SAP Gateway Flight service and our SQL Anywhere databse running in OpenShift.

 

1. Right-click on the Demo project and select New->Teiid Metadata Model. For Model Name enter AllFlightDataModel. Set the Model Type as View Model and click Finish.

2. Right-click on the new model and select New Child->Table. Enter AllFlightDataTable for the Name and click OK. Double-click on the generated model to open the modeling diagram.

AllFlightDataDiagram.png

3. Now expand the SAPGatewayFlight model and drag the FlightCollection table to the arrow with the T in the diagram. This will add the FlightCollection table to the transformation. Now do the same with the liveFlightFeed table form the myflight model to complete the transformation. Save your changes.

allflighttransformation.png

4. In order to expose the view table as an OData service in Data Virtualization, we need to add a primary key. Right-click on the AllFlightDataTable and select New Child->Primary Key. Name the new Primary Key allFlightData_PK and click the Ellipsis for the Columns property in the properties tab below the Model Explorer. Select the airline_iata, depart_time, arriveTime, flightDate, fromAirport and toAirport columns and click OK Save your changes.

 

We are now ready to create and deploy our virtual database.

 

1. Right-click on the Demo model project and click New->Teiid VDB and add your models.

newvdb.png

Click Finish.

2. To deploy your new VDB, right-click on the Flight VDB in the Model Explorer and select Modeling->Deploy. A new datasource will be created for you and your Flight VDB will now be available in OpenShift as an OData service. You could also create a JDBC connection to the VDB if you wanted to, but we will use the OData capabilities of Data Virtualization for our Android application.

 

Testing Your New Service

 

Let's test our new service running in OpenShift. The URL for an OData service running in Data Virtualization is http://{host}:{port}/{vdbName}/{modelName.tableName} so we will use http://localhost:8080/odata/flight/AllFlightDataModel.AllFlightDataTable?$format=json to see our federated SAP data.

Note: Make sure you have port forwarding running in order to execute the service with the above URL.

5. Run the Android application

 

Running the Android app in JBDS using the Android emulator, you can access your data in DV on OpenShift using OData urls.

I recently upgraded my Mac to Mavericks and, of course, MySQL was hosed afterwards. I use MySQL a lot for development and demos and dreaded going through the pain of getting it running again. I recalled the initial installation was fairly unpleasant, with a bit of trial and error before I finally had my test databases the way I wanted them. When I upgraded to Mavericks, I panicked when my tests started failing with errors connecting to MySQL. Alas, it was broken. A bit of Googling indicated that it needed to be reinstalled on Mavericks, which was not actually true. Had I seen this first, I would have known better. Anyway, I came across this Apple User Tip that not only made installation and setup a snap, but made integration with OSx even better. Here is a summary from the article:

 

1. Download MySQL, if you haven't already. The article indicates to download the OSx 10.6 pkg with the dmg. I wanted the latest, so I got mysql-5.6.17-osx10.7-x86_64.tar.gz and extracted to /usr/local/mysql. Either way seems fine though.

2. Run sudo vi /Library/LaunchDaemons/com.mysql.mysql.plist , add and save the following:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

  <dict>

    <key>KeepAlive</key>

    <true/>

    <key>Label</key>

    <string>com.mysql.mysqld</string>

    <key>ProgramArguments</key>

    <array>

    <string>/usr/local/mysql/bin/mysqld_safe</string>

    <string>--user=mysql</string>

    </array>

  </dict>

</plist>


3. Run sudo vi /etc/my.cnf, add and save the following:

[client]

socket=/var/mysql/mysql.sock

 

[mysqld]

socket=/var/mysql/mysql.sock


4. Run sudo launchctl load -w /Library/LaunchDaemons/com.mysql.mysql.plist and MySql will start. You are then ready to set the root password and add any users.


Hopefully this will save you some time. There are lots of things like this that can be a pain to do (or remember what you did last time). As I come across those, I will add them here.


BlN6FL7CYAAvtnz.jpg

There are so many things to love about Red Hat Summit.. learning about new features and technologies, meeting current and prospective customers, seeing colleagues face-to-face after only knowing them virtually for years, the keynotes (especially the middleware keynote and demo), and presenting topics I am passionate about to people that are eager to listen and learn. That last one is my favorite. This year I spent a lot a time at the Red Hat JBoss Integration pod. There I could educate people on the power and value of Data Virtualization where we had looping demos on Data Virtualization (DV) with big data and SAP NetWeaver Gateway. Attendees were drawn by the lure of big data and intrigued by versatility of Data Virtualization that could not only consume Hadoop and other big data sources, but also integrate them with other sources for a single centralized view of all their analytical data.

BlYuhVRCMAAHVNj.jpg

 

The last few years, I have been involved with SAP integration with JBoss Middleware. Summit gave me a platform to show-off consuming SAP NetWeaver Gateway services into DV and exposing as a relational source for federation with other disparate but related data. Not only does this demo well, it illustrates the ability to easily consume SAP data and combine with other important data to create a single unified view of data that can be consumed by enterprise applications via JDBC, ODBC, SOAP, REST or OData. Here is a recording of the demo:

 

 

I also gave a joint talk with my Red Hat colleagues Kenny Peeples and William Collins that covered all connectivity options for SAP with Red Hat JBoss Middleware. I gave a brief history of solutions, starting with SAP Enterprise Services Registry which lead to the introduction of SAP NetWeaver Gateway in October of 2011. I also talked about OData which is now an OASIS spec with the recent approval of v4 and showed a demo of Data Virtualization with SAP. Kenny covered JBoss Fuse with the new SAP NetWeaver Camel component and William discussed and demo'd his new JCo Camel component. There was a lot of interest in the talk and many great questions. I have attached the slide deck from the presentation to this blog.

 

I'm looking forward to next June in Boston for the next Summit! Hope to see you there!

Ted Jones

OData v4 - The Journey

Posted by Ted Jones Mar 17, 2014

The OASIS OData TC Experience

I was privileged to represent Red Hat on the OASIS OData Technical Committee. Thanks to the tireless efforts of committee members from Microsoft, SAP and others, the end result is a powerful and flexible specification that will revolutionize REST services. There is much more work to do as the committee turns its focus to v4.1 and v5, but now is a good time to reflect on the awesome job done by the group!

 

Red Hat has been a proponent of OData and we have incorporated v2/v3 support in our projects and products. Our Data Virtualization product (based on Teiid), has extensive support for OData. Inspired by the SAP NetWeaver Gateway server, we added the ability to import Gateway services into a virtualized view for seamless integration of SAP data with other disparate data sources.

 

This led to full support for generic OData services as well as creating an OData provider for all data deployed to the Data Virtualization environment. See here for an example of how to consume and produce OData services in Data Virtualization.

 

So what's new in v4?

Lots! This document does a great job in identifying deletions, additions and improvements in v4 compared to v3:

New in OData v4

 

Will Red Hat Support v4?

You bet! We are excited to add the capabilities of the v4 spec into our tooling and Data Virtualization engine. There are two open source projects put forward by SAP which we plan to contribute to and support. Apache Olingo and Eclipse Ogee will fit nicely within our projects and Eclipse tooling.

 

Want to Contribute?

Are you or your company interested in shaping the next version of OData? You are more than welcome to join the committee and help us make a great spec even greater!

Ted Jones

Red Hat Summit 2013

Posted by Ted Jones Jun 17, 2013

I am back home from Boston this week after my first Red Hat Summit. What a tremendous conference this was... very well run and attended. I am really proud to be counted as a Red Hat employee.

 

The purpose of my attendence was to co-present with Mustafa Saglam from SAP to talk about our co-ordinated efforts over the last 18+ months. The presentation was called Interoperability Results from SAP & Red Hat Collaboration and featured an overview of SAP NetWeaver Gateway as well as demos of integration with JBoss Middleware, specifically with JBDS and Red Hat JBoss Data Services. Getting a chance to show customers and potential customers how to consume SAP data in JBoss using an OData, REST-based, open standards interface exposed by SAP NetWeaver Gateway was an exciting opportunity for us and was well-received by the audience. Many companies have struggled to find ways to consume or integrate their SAP siloed data and the collaboration between Red Hat and SAP has removed that barrier. The demos walked people through how to take an SAP NetWeaver Gateway OData service and import as a data source in JBDS/EDS. I also showed off our new OData Translator that maps OData services to relational entities and generates queryable tables in Teiid Designer. These tables can then be deployed in a virtual database (VDB) and accessed as a JDBC or REST source. Finally, I showed how any deployed VDB can be accessed as an OData service using the OData provider capablities of Red Hat JBoss Data Services.

 

photo 3.JPG

 

While I was there, I also worked the Data Services demo pod. This gave me a chance to share the power of data virtualization with attendees. Many had never heard of our data services product or even knew such a technology existed. I had many tell me their data integration issues and it was so rewarding to see the light go on in their head when they realized that we could not only solve their dilema, but we could do so at a fraction of the cost and time that it would take to build a custom solution. The SAP booth was right next to the data services pod, so the close proximity allowed us easily share our stories with attendees. We both walked interested parties back and forth to share the exciting and complimentary technologies both SAP and Red Hat have developed.

 

The conference also gave us an opportunity to announce a new Fedora 18 virtual image that is pre-configured to allow for the installation of the SAP NetWeaver Gateway Trial Bundle. The bundle allows you to take Gateway for a spin and even comes with a ready-to-go OData service that contains flight information. You can use this Gateway instance to try out the JBoss integration or just get familiar with OData.

 

I am excited to continue working with Mustafa and SAP on developing more integration with JBoss middleware. Stay tuned for more technical advances between us to make your SAP and Red Hat integration projects a snap!

The SAP Services Registry in SAP is a UDDI v3 compliant web services registry that allows developers to access underlying SAP data. With these services, it is possible to generate client stubs and create applications that utilize this data. SAP has come out with an open source plugin that allows searching a registry and getting the associated WSDL document for a given service as well as deriving other UDDI based information. Please see this video for an example of how to use the plugin in JBoss Tools 3.3.

 

As the video describes, there is a conflict with the search plugin and the EGit plugin. As a result, the two cannot run in the same environment. Once this issue is resolved, it is my end goal to include the plugin in JBoss Tools. The source code can be found here: https://github.com/tejones/sap-services-registry-eclipse

 

Patches welcome.

In JBoss' continuing partnership with SAP, we are creating more and more ways for our products to integrate. With the advent of SAP Netweaver Gateway, there are even more ways to consume and use SAP data within JBoss and Red Hat solutions.

 

SAP NetWeaver Gateway is an OData server that exposes SAP data via RESTful services using standards such as HTTP, AtomPub and JSON. This is a perfect integration point for many JBoss technologies. For example, using our Enterprise Data Services (EDS) product built on Teiid, we can consume a Gateway service and federate the data from that service with other related information from disparate data sources. A federated view in EDS means we can bring together all of the data we care about and expose it as a single source, either relationally or as a REST service. If we wanted to, we could even combine mutliple SAP Gateway services and create a single source. COOL!!!

 

Check out this example of how to accomplish this integration in the cloud using SAP Gateway, EDS, OpenShift and a MySQL instance: Example

 

The transformations required to create the federated view are generated using a wizard in Teiid Designer 7.8. You can see a tutorial video on importing the SAP Gateway source and generating the required transformations HERE.

 

Enjoy and stay tuned for more tech goodness from Red Hat and SAP!

"I don't want to make this a great conference, I want to make this the best conference you have ever been too.". Those were the words of StrangeLoop founder Alex Miller at this year's StrangeLoop conference in St. Louis, MO. I am here to say that, at least for me, Alex reached his goal.  I have been to many conferences over my 17 years working in information technology and none have had such an array of compelling speakers and sessions. The list of speakers read like a "who's who" of software superstars. 

 

Just to name a few..


Gerald Sussman

Allen Wirfs-Brock

Rich Hickey

David Geary

Howard Lewis Ship

 

Highlights

 

  • Chatting with Gerald Sussman while he was sitting at my table during Allen Wirfs-Brock talk about the post-pc computing era, or the "Ambient Computing Era" as he called it.
  • Mike Lee's talk on Product Engineering. I  want to move to Amsterdam.
  • David Geary's HTML5 presentation. I now know everything I need to know about Canvas. Very powerful stuff and David presented it very well. If you ever get a chance to see him present, do it.
  • The Language Panel
  • Neil Ford's presentation on Functional Thinking. Ok... I get it now. Thanks Neil!
  • Rich Hickey's Simple Made Easy talk.  There is no room for "complect" code in our software!
  • Lance Ball's session on Datamapper - an ORM for Ruby. An impressive suite of technology including TorqueBox and Infinispan.
  • Mingling and exchanging ideas with my peers in the industry. This conference is a draw to some of the finest talent in software development and architecture in our field (over 900 attendees this year)... that speaks volumes to the quality product Alex put together.

 

The only problem I had with this conference was that there were so many great sessions that I could not attend. I had to pick one and miss out on the others. The good news is all the sessions were recorded and will be posted on infoq.com within a few weeks.

 

Looking forward to next year since, by Alex's definition, it will be my new best conference EVER! I have no doubt he will pull it off.

Filter Blog

By date:
By tag: