Relating Systems Integration to Knowledge Management (and other classes)

So I'm sitting in Knowledge Management class now, where Dr. Stucke is proctoring for Dr. Gallivan, who is stuck in China somewhere.  We're about to watch a YouTube video, so I figured this is a good time to blog instead of taking my customary nap.

I'm trying to link the Systems Integration class with the Knowledge Management class in my mind, maybe somebody has some ideas.

First, my definition of Knowledge Management (KM) is the art/science of linking up knowledge seekers with knowledge keepers.  This can be done through knowledge management systems (codification of knowledge, usually knowledge stored in a database, etc.), social knowledge management programs (personalization of knowledge, this can include email, conferences, IM), or knowledge in the community (like in blogs and social networking).

I think KM relates the most to Systems Integration ideas when the KM codification approach is used.  Here's how I think we can link KM with SI:

Let's say we want to make a database driven Intranet to hold and store knowledge, and we want to link or integrate the KM database with an employee database, perhaps using ETL on way to a data warehouse.  Well, whenever you are linking two databases, using ETL, or a data warehouse, we have moved into the realm of Systems Integration!

I was blogging to Asha about this on another posting - that really no matter what you're doing in Information Systems, that Systems Integration plays an important role.  Without integrating systems, you just have information systems silos that may be important by itself, but silos are very weak when information or data needs to be shared.

I never considered Systems Integration before this class, but now I see it everywhere, and it feels like a fog has been lifted.  Now the next time I'm dealing with a system or thinking about implementing a new system, I will think of how it will possibly need to be integrated later.

Anyway, the YouTube video is ending, I have to pay attention now.  Does anyone have any thoughts of how Systems Integration relates to their other classes?  Thanks!

Shibboleth

I did small researches on Shibboleth in Wiki and Shibboleth's web site :

Shibboleth (pronounced /ˈʃɪbəlɛθ/ or /ˈʃɪbələθ/) is any distinguishing practice which is indicative of one's social or regional origin. It usually refers to features of language, and particularly to a word whose pronunciation identifies its speaker as being a member or not a member of a particular group. The term originates from the Hebrew word "shibbólet", which literally means the part of a plant containing grains, such as an ear of corn or a stalk of grain or, in different contexts, "stream, torrent".

Shibboleth® System:

The Shibboleth System is a standards based, open source software package for web single sign-on across or within organizational boundaries. It allows sites to make informed authorization decisions for individual access of protected online resources in a privacy-preserving manner.

Because I have some library cataloguing experiences, this method reminds me of Dewey decimal classification. But under the name I can see that it uses a method similar to Object-oriented design to manage those individual accounts. Using this method, the account management will be able to manage the accounts more effectively and efficiently. Last September I joined a presentation sponsored by Deloitte Consulting. They introduced a role-based method to manage the identities in the industries and used a similar way to present how to manage the accounts. How is Shibboleth related to this role-based identity management? Perhaps we will ask our speaker again.

With the coupling of more technologies, Identity Management becomes more urgent not only in academia but also in industries. In ING, every year we test access controls to a large amount of financially significant applications. Without appropriate management on these access controls, the company or the university might incur great financial risk and reputational risk.

Systems Integration Philosophy - lessons learned from this class

I want to share some random thoughts I have about Systems Integration while sitting on the couch eating M&Ms.  Here are some things I've learned that will forever change my view on integration.

My first thought is that this is an extremely complex subject.  I never thought about systems integration much before coming into this class, but after taking this class and learning about the myriad of technologies available, it seems to me that no one can know much about all of these technologies, and the more you know the more superficial the knowledge is across the board, as the technologies are so varied and different.  In fact, I think you can make a whole undergrad or grad degree called MS Systems Integration, for example, and that still wouldn't be enough.

Also, as new and newer technologies emerge, you need more and more techniques for integrating systems, which is sort of a rehash of what I just said, but from a different angle.  The ERP is a relatively new advent, and think of all the technologies associated with it.  Think of all the integration technologies involving the web, databases, servers, etc.  Because there are so many new technologies, it's hard for anyone to keep up with them.  So like most technologies, specialization becomes very important.  So you can be a database integration expert, an application integration expert, an ERP implementer, but I doubt you can be an expert in all the kinds on integration, unless you're just an exceptional superstar.

Another thought - where would the world be without integration?  What good would all these technologies be if they cannot interoperate?  I always thought of technologies as more of standalone islands, but the key is getting them all to talk in a meaningful way.

Another - what does the future hold?  I wonder if the trend will continue of the technologies becoming more and more sprawling, or if we'll start to see more standards and fewer ways to have to integrate systems.  Will the integration of systems become more graphical, automated, and intuitive?  Or will they become more flexible and require coding?  Will they rely more on things like APIs where you don't need much back-end knowledge?  Will there more of a push to open source integration?  As Microsoft loses more and more of its grip on future market share, will systems integration become yet even more complex?  Is there going to be a large vendor that specializes in just integration?

Wow, it's enough to make your head spin.  I really enjoyed learning about all these, but the more I learn, I realize I know less and less.  But the first thing to learning is knowing what you don't know, realizing what's out there, and then you can pick and choose what fits your needs and become expert at it - and go from there.

Are there any other philosophical thoughts on systems integration out there? Am I right in assuming the present is extremely complex to warrant high specialization and even its own degree?  Will the future become even more or less complex?  As I finish my bag of M&Ms, I'm curious to know what you think.  I have a free bag of M&Ms to the best response.  Blog on!

CIS 8020 Assignment 2 FF Google Chart API - Mega Hot Pot Financial Group

The Scenario Mega Hot Pot Financial Group is a start-up insurance company in the South East. Recently it announced its purchase of 5 financial applications to manage its retail annuity business which is worth thousands of dollars. Risk Management manager, Susan builds a group of 3 Risk Assessment Analysts (RCA) working on controls established for these financial applications. They are required to follow Susan's work process perform their control testing. Susan arranges a Friday meeting for the new group and also invites her boss to the meeting.

This Friday morning Susan found that her chart couldn't be opened 10 minutes before the meeting. Luckily her assistant Mary happened to know how to create a chart from scratch by using Google Chart API. According to Susan's requirements, this chart will mark the work process for the team. It will mark the start process, the end process and each step with the percentage. It will visualize the process, standardize the process and help the team management manage their time and resources under each control test. Additionally, this chart will be easily generated, shared by everyone, and easily updated by each team member if there is some variance on the percentage phase by phase.

A Potential Solution Google provides a useful Chart API (Application Programming Interface) which can be accessed to quickly and easily produce highly customizable charts. The Google Chart API lets users generate line, bar, pie charts within a web page. The user can specify parameters such as color, size and titles. Mary created this chart by defining its type, its parameters for data and labels, its size, its title and its color in Google .

The Prototype Here is the chart prototype:

http://chart.apis.google.com/chart?cht=p3&chd=t:5,5,15,5,20,25,15,5,5&chs=550x200&chl=Start>>>Send requests(5%)|Receive the population(5%)|Request change tickets(15%)| Collect change tickets(5%)|Document the test table(20%)|Document the work paper(25%)|1st review(15%)|1st review approval(5%)|End>>>2nd review approval(5%)&chtt=IT+RCA+Control+Testing+Work+Chart|By+Mary&chco=ff0099

The Reason Why This web API is chosen for implementation because:

1. It is easier to generate

2. It is shared by all the team members

3. It can be updated by all the team members

4. It meets their requirements including marking the start, the end, the work process and the work step percentage

5. It provides a dashboard like chart visualizing their work process

This prototype is ideal for users who don’t want to buy complex charting solutions like Mega Hotpot. Other charting solutions like Excel can achieve the goal. In this scenario, Excel can't be used, so they have to build the chart from the scratch.

There are other solutions building the chart. They include:

1. Insert the chart JPG after creating a chart in a software(e.g. Excel)

2. Input some data into a database and create a chart by using data analysis tools (e.g. Crystal Reports)

But these two solutions have their drawbacks compared with the Google Chart API:

1. The process is complicated and long to create such a chart

2. The user should know how to know how to input the data into a database and how to use the tool

CIS 8020 Assignment 2 MM Google Calendar API

This blog entry shall discuss my implementation of the Google Calendar API to solve a particular business problem.

1. Introduce the scenario or the case.

Morgan Computer Consulting has a big client (Client Alpha) that requires a new ERP implementation. Morgan Computer Consulting has formed a 20-person international project team to work on this big project. Coordination among team members and project managers will prove to be essential in getting the project done efficiently.

The team needs a way to easily coordinate meetings, installations, testing, etc. Morgan Computer Consulting doesn't want to have to buy any new software or go through the trouble of installing bulky software on employees' computers. Morgan Computer Consulting needs an easy to use, cloud-based, free calendar system that all global project team members can view in order to stay coordinated and connected.

2. Explain how you can use one of the Google APIs for a potential solution.

Google Calendar offers a very easy to use API that solves the project team's needs. Google Calendar offers the flexibility of easily embedding within a dashboard or any web page (using an iFrame), and allows the customization of allowing users to see daily, weekly, monthly, or agenda item views. Every team member can see the calendar at the same time, and the calendar can be controlled by specific people (one or multiple) such as the project managers simply by logging into Google and giving Calendar permissions. The Google Calendar is free and easy to use, features an easy print option, and allows the global project team to view all project team calendar items.

3. Use Google API to build a prototype. The output can be directly integrated to the same blog entry; or you may need a separate HTML/Script gadget.

Here is the implementation of the Google Calendar. Some sample calendar entries have been entered:



4. Explain why you chose this web API for implementation; if possible, compare to other possible solutions.

This particular API's huge advantage is that it's extremely easy to implement and customize. Other advantages are that it's free and visibly aesthetic. Other solutions would be Microsoft Outlook or Microsoft Sharepoint, neither of which are free or particularly easy to use. Those products would provide too much bulk and overhead to be an effective, efficient solution.

Google Calendar offers real-time coordination without the need for backup or maintenance. Printing is very easy using Google Calendar, and can be easily embedded in an iFrame for easy web, dashboard, or mashup implementation. It's a great solution for this particular problem.

CIS 8020 Assignment 2 BC Google chart API--Sinopec stock valuation

China Petroleum & Chemical Corporation (Sinopec) is one of the largest petroleum companies in China. Sinopec has recorded a strong financial performance in recent years. However, because of over high crude oil price, the company suffered loss in its 2008 fiscal year. Still, many portfolio managers desire to invest in Sinopec during 2009 and they need to take a look at its recent financial data to valuate its stock market performance.

Google Chart API is a good candidate solution for this problem. Google API chart generates bar chart as well as line chart images which shows the company's yearly and first quarter net profit and Earnings per share(EPR) data. The charts is prototyped as following:







Based on the Google chart API, we can see that the company’s annual and first quarter net profit is highly correlated with its Annual and first quarter EPS. The graph shows that its first quarter EPS increased in FY 2009 compared with that in FY 2008. As we can see, this company is a valuable stock to invest in now as part of the portfolio.

Google API charts are much more straightforward and illustrative than examine Sinopec's financial statement and balance sheet for their profit analysis. This application can be nicely integrated to the website as well.

Here I attached the video that Dr. Zheng suggested to upload in our blog. This video gives a very good example helping us to understand how abnormal data is analyzed in the existing database using SSIS' data profiling function.

MegaHotPot's presentation handout:

This is a digital version of our handout for tonight.  Hope you guys enjoy it:

Abstract and Reading for Today's Topic: The ETL process in Data Warehousing

Abstract of Presentation:

Data warehousing is one approach for data integration. Today, Our's topic is mainly focusing on ETL process for data warehousing.We would start with introducing data integrations approaches. One way to another, data warehousing is one of the most commonly used data storage methods used by data integration system designers. Advantaged and Drawbacks of data warehouse will all be covered in our presentation.

ETL is the process used in data warehousing. What ETL does, the issues within ETL and what's the general solutions to the problems would lead us to a practical world for data warehousing. SSIS is a platform for building enterprise-level data integration and data transformation solutions. Its features and development will also be discussed in our presentation.

Last but not least, most interesting part is from Mike's demostration! Please stay tuned and enjoy.

Reading materials suggestions:
1. Take a look at this website: http://www.datawarehouse4u.info/
It shows how ETL work in data warehouse, which could give you a intuitive view of data integration process within data warehouse.
2. http://communication.howstuffworks.com/data-integration.htm
This would be your startpoint in understanding data warehouse from data integration perspective. How data warehouse is better than other approached? Check it out.
3. Data Cleaning: Problems and Current Approaches. This article is for data cleaning and ETL, which is suggested by Dr, Zheng. It's a super important paper, highly related to our topic and presentation materials.
4. Another one suggested to read before our presentation would be:High impact Data Warehousing with SQL Server Integration Services and Analysis Services.(take a look at first few slides).

There are many other materials we've read for preparing our presentation. We included them in our presentation slices(the last slide as bibliography). Please check them out for further your further study.
(thanks bing, minor edits - mtm)

Data Warehouse

Thanks to Dr.Zheng's suggestions on our presentation! And we will be able to focus more on a new topic on Data Warehouse. I did some preliminary researches on Data Warehouse and found that Data Warehouse is closely related to an important component of ERP systems - Online Analytical Processing (OLAP). Luckily, one of the groups provided us a vivid presentation on ERP system at the beginning of this semester. Based on this knowledge, we will be able to expand further on OLAP.
OLAP includes decision support, modeling, information, retrieval, ad hoc reporting/analysis, and what-if analysis. Data Warehouse plays an important role in supporting business analysis applications. It is called a meta-database constructed for quick searching, retrieval, ad hoc queries, and ease of use. The data are normally extracted periodically from an operational database or from a public information service. Data Warehouse is a relational or multidimensional database that may consume hundreds of gigabytes or even terbytes of disk storage. Once the data is loaded into the warehouse, data are accessible via various query and analysis tools that are used for data mining.
The primary reason for using a data warehouse is to optimize business performance. By providing customers and suppliers with the information they need when they need it, the company can improve its relationships and provide better service, e.g. a responsive and efficient supply chain. Hope we will find a good example to explain the importance of Data Warehouse in our presentation.
Implementing a successful data warehouse involves installing a process for gathering data on an ongoing basis organizing it into a meaningful information, and delivering it for evaluation.
Dr.Zheng's links are also added here for sharing:
http://en.wikipedia.org/wiki/Extract,_transform,_load
http://www.information-management.com/infodirect/2009_109/10014988-1.html

For the Anti-XML crowd, sometimes you have no choice!

The below link is the National Vulnerability Database XML mapping that major security vendors use, including my old company IBM ISS.

http://nvd.nist.gov/download.cfm#CVE_FEED

Basically, NVD (through a committee decision) has decided to use XML mapping to describe vulnerabilities and the various platforms they affect. Security vendors would then import this information into their databases and map them to their own security vulnerability records.

It's confusing, I know, but the bottom line is sometimes a third party (such as the government in this case) forces companies to read, manipulate, and use XML data. So if you're in the anti-XML camp, sometimes outside influences box you into a corner and you have no real choice. Bummer, eh Phil? :)

Readings on SQL server integration services

Here are two books related to SQL server integration services and they provide comprehensive materials on SQL server integration services. For people interested, you can access to the free book web site above.

1. Microsoft SQL Server 2008 Step by Step by Mike Hotek, Microsoft Press © 2009 (544 pages)

This learn-by-doing tutorial teaches you the essentials for getting started, from data transformation to scripting, to security, and best practices, and so on.

2. SQL Server 2005 Bible by Paul Nielsen

This book includes all the basics, best practices, dozens of targeted examples, and sample code, this comprehensive tutorial is a must for any developer or database administrator transitioning to the new version of SQL Server.

More readings and ideas will be published before our presentation day! Stay tuned.

Handout for presentation - presentation thoughts

Hi everyone,

Dr. Zheng has asked us to publish the little handout that we handed out for our DB middleware presentation last week, and here it is:

http://docs.google.com/View?id=dfpw84nc_7cbr494f3

I think it would be useful if the groups import PPTs into Google Present (part of google docs) and then publish them so everyone can view the presentations before/after the presentations.  I think this is useful because sometimes I want to go back and view a point that one of the groups made, but it's impossible if the PPT isn't shared.  It just takes 2 minutes to upload and share your PPT or Word file to Google docs, and then publish it and post the link.

While receiving my BS in Technical and Professional Communication many years ago, we learned that often times it is helpful to hand out notes and handouts to strengthen and diversify a PPT presentation.  I think Google Present is a good way to avoid having to handout a bunch of paper, but still save your presentation for later use.  What do you guys think?

Thanks! -mtm

MegaHotPot Readings Blog and Abstract for Upcoming Database Middleware Presentation

(Readings Blog and Abstract combined per Dr. Zheng's request - mtm)


Well, the MegaHotPot presentation is coming right up on Tuesday, and we're hammering out the last details.  We hope to live up to the great presentations that we've already heard from, and equally hope to set the standard for future presentations.

For those who would like to do some readings before the presentation, we have presented six readings in two categories below - 3 from scholarly journals (free PDFs) and 3 casual readings (from well-known web sites).  

The scholarly journal readings are quite dense, while the casual readings are very light.  Taken together, they give a good overview, good examples, and technical details of database middleware.  Give them a once over and see if you agree.

Scholarly Journals:

Transforming Heterogeneous Data with Database Middleware: Beyond Integration

http://dbs.informatik.uni-halle.de/Lehre/Integr/stuff/debull99.pdf

--- For those who want a highly technical read, with a few graphics thrown in for good measure.  It's quite a dense read, but it's just six pages so it's not overly exhausting.  Good article.

Grid Enabled Relational Database Middleware

http://hep.ac.uk/papers/GGF3Rome2001.pdf

---DataGrids as they relate to DB Middleware discussed here.  Also pretty technical, but again not very long.  These best part about this article are the different examples they use to show DB middleware in action.

Web Client and ODBC Access to Legacy Database Information: A Low Cost Approach

http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=2233436&blobtype=pdf

--- This is an older article, but it's just five pages and sums up a lot of ODBC/Legacy issues very nicely.  Definitely Worth a Read.

Casual Reading:

What Is Middleware? Web site

http://www.middleware.org/whatis.html

--- Great starting place to learn about middleware, and has a nice little section on DB middleware, too.

Open Database Connectivity - Wikipedia

http://en.wikipedia.org/wiki/Open_Database_Connectivity

--- I'm a wikipedia fan, and this nice summary article shows why.  Talks about ODBC and how it relates to other DB middleware types.  If you don't know much about ODBC, give this a read.

ODBC Microsoft Article

http://support.microsoft.com/kb/110093

--- ODBC talk straight from Microsoft.  This reads a little like a sales pitch, but I think it's a good basic coverage on ODBC that takes the above Wiki article and extends even further.  The funniest header in this article is "What's So Great About ODBC?"
Classic Microsoft. :)

_____
Abstract of presentation, as presented by Bing:

Abstract of presentaion--Database Middleware
In our presentation, we would start from basic introduction of middleware, different types of middleware,and then dig into our main topic, which is database oriented middleware. we would both provide detailed explanation of different kinds of database middleware and empirical demonstration of one database middleware, which is ODBC.

You would actually learn how database middleware works and differenciate database middleware from other middlewares. With ODBC demo, you would know intuitively how database middleware work and how to connect databases or applications via database middleware.

Our presentation is the second one on week 5. Hope everyone enjoys!

Here is the tentative link through our google presentation: http://docs.google.com/present/edit?id=0Ae7CQJaENDlQZGNuc3g5MndfMjM4Y214Njg3ZjU&hl=en

Are you interested in Middleware? Go and check our blog!!

Our Middleware presentation is around the corner. To avoid dry out your interests, I would love to provide a little background info on this topic.
Middleware works as "glue" between the operating system and the application. It allows one application interoperate with another software. No programmers need to understand internal process. Isn't that neat?
I found two youtube videos which shows how Middleware works and its use in RFID. Enjoy!

How Middleware eliminates your downtime


The importance of Middleware in RFID

A simple SAP/ERP video.

Since we heard a great presentation on ERP as it relates to systems integration tonight from Asha, Jimmy, and Rahul, and also since SAP was featured in the presentation, I thought it'd be relevant to post this youtube video for those who are unfamiliar with ERP. This video gives quite a nice overview to SAP and is under five minutes:



Videos like this are abundant, and you can search for some that go into quite a bit of detail in showing screenshots and some how-to's.

I, like Dr. Zheng, have never really used an ERP product, although a client of mine once had something called Solomon and they were very unhappy with it. The client had it before I got there, and they spent a ton of money on it and it barely worked.

I'm not a big fan of ERP, as statistics show (from CIS8010) that about half of ERP implementations fail, and ERP can negate any competitive advantage that a company has. In my opinion, the upfront cost and the risk involved is just too much to put all your eggs in this basket. But it seems to be a reality in business today, so we should definitely be aware that it exists and know something about it. 

Great job on the presentation tonight, guys, enjoy the video!
-mtm

Our first presentaion is on week 5--Interface Middleware

Here is the basic knowledge of materials that we might be involved in our presentaion.

Interface middleware:
Middleware is computer software that connects software components or applications.

RPC(remote procedure call):
It is anInter-process communication technology that allows a computer program to cause a subroutine or procedure to execute in another address space (commonly on another computer on a shared network) without the programmer explicitly coding the details for this remote interaction.

API(application programming interface):
It is an interface in computer science that defines the ways by which an application program may request services from libraries and/or operating systems.

We would pick a specific topic from these three major areas and it would be updated before our presentaion.

Besides, there is one good web that mainly focus on RPC. we would link it in our blog.

Security Overview (Integration Services)

Security in SQL Server Integration Services consists of several layers that provide a rich and flexible security environment.

These security layers include the use of digital signatures, package properties, SQL Server database roles, and operating system permissions. Most of these security features fall into the categories of identity and access control.

SQL Server Integration

Integration Services will be used to solve complex business problems by copying or downloading files, sending e-mail messages in response to events, updating data warehouses, cleaning and mining data, and managing SQL Server objects and data. The packages can work alone or in concert with other packages to address complex business needs. Integration Services can extract and transform data from a wide variety of sources such as XML data files, flat files, and relational data sources, and then load the data into one or more destinations.

Integration Services includes a rich set of built-in tasks and transformations; tools for constructing packages; and the Integration Services service for running and managing packages. You can use the graphical Integration Services tools to create solutions without writing a single line of code; or you can program the extensive Integration Services object model to create packages programmatically and code custom tasks and other package objects.