This is a digital version of our handout for tonight. Hope you guys enjoy it:
Blog Votes and Statistics
16 years ago
Tuesday, October 13, 2009
Abstract and Reading for Today's Topic: The ETL process in Data Warehousing
Abstract of Presentation:
Data warehousing is one approach for data integration. Today, Our's topic is mainly focusing on ETL process for data warehousing.We would start with introducing data integrations approaches. One way to another, data warehousing is one of the most commonly used data storage methods used by data integration system designers. Advantaged and Drawbacks of data warehouse will all be covered in our presentation.
ETL is the process used in data warehousing. What ETL does, the issues within ETL and what's the general solutions to the problems would lead us to a practical world for data warehousing. SSIS is a platform for building enterprise-level data integration and data transformation solutions. Its features and development will also be discussed in our presentation.
Last but not least, most interesting part is from Mike's demostration! Please stay tuned and enjoy.
Reading materials suggestions:
1. Take a look at this website: http://www.datawarehouse4u.info/
It shows how ETL work in data warehouse, which could give you a intuitive view of data integration process within data warehouse.
2. http://communication.howstuffworks.com/data-integration.htm
This would be your startpoint in understanding data warehouse from data integration perspective. How data warehouse is better than other approached? Check it out.
3. Data Cleaning: Problems and Current Approaches. This article is for data cleaning and ETL, which is suggested by Dr, Zheng. It's a super important paper, highly related to our topic and presentation materials.
4. Another one suggested to read before our presentation would be:High impact Data Warehousing with SQL Server Integration Services and Analysis Services.(take a look at first few slides).
There are many other materials we've read for preparing our presentation. We included them in our presentation slices(the last slide as bibliography). Please check them out for further your further study.
(thanks bing, minor edits - mtm)
Data warehousing is one approach for data integration. Today, Our's topic is mainly focusing on ETL process for data warehousing.We would start with introducing data integrations approaches. One way to another, data warehousing is one of the most commonly used data storage methods used by data integration system designers. Advantaged and Drawbacks of data warehouse will all be covered in our presentation.
ETL is the process used in data warehousing. What ETL does, the issues within ETL and what's the general solutions to the problems would lead us to a practical world for data warehousing. SSIS is a platform for building enterprise-level data integration and data transformation solutions. Its features and development will also be discussed in our presentation.
Last but not least, most interesting part is from Mike's demostration! Please stay tuned and enjoy.
Reading materials suggestions:
1. Take a look at this website: http://www.datawarehouse4u.info/
It shows how ETL work in data warehouse, which could give you a intuitive view of data integration process within data warehouse.
2. http://communication.howstuffworks.com/data-integration.htm
This would be your startpoint in understanding data warehouse from data integration perspective. How data warehouse is better than other approached? Check it out.
3. Data Cleaning: Problems and Current Approaches. This article is for data cleaning and ETL, which is suggested by Dr, Zheng. It's a super important paper, highly related to our topic and presentation materials.
4. Another one suggested to read before our presentation would be:High impact Data Warehousing with SQL Server Integration Services and Analysis Services.(take a look at first few slides).
There are many other materials we've read for preparing our presentation. We included them in our presentation slices(the last slide as bibliography). Please check them out for further your further study.
(thanks bing, minor edits - mtm)
Monday, October 5, 2009
Data Warehouse
Thanks to Dr.Zheng's suggestions on our presentation! And we will be able to focus more on a new topic on Data Warehouse. I did some preliminary researches on Data Warehouse and found that Data Warehouse is closely related to an important component of ERP systems - Online Analytical Processing (OLAP). Luckily, one of the groups provided us a vivid presentation on ERP system at the beginning of this semester. Based on this knowledge, we will be able to expand further on OLAP.
OLAP includes decision support, modeling, information, retrieval, ad hoc reporting/analysis, and what-if analysis. Data Warehouse plays an important role in supporting business analysis applications. It is called a meta-database constructed for quick searching, retrieval, ad hoc queries, and ease of use. The data are normally extracted periodically from an operational database or from a public information service. Data Warehouse is a relational or multidimensional database that may consume hundreds of gigabytes or even terbytes of disk storage. Once the data is loaded into the warehouse, data are accessible via various query and analysis tools that are used for data mining.
The primary reason for using a data warehouse is to optimize business performance. By providing customers and suppliers with the information they need when they need it, the company can improve its relationships and provide better service, e.g. a responsive and efficient supply chain. Hope we will find a good example to explain the importance of Data Warehouse in our presentation.
Implementing a successful data warehouse involves installing a process for gathering data on an ongoing basis organizing it into a meaningful information, and delivering it for evaluation.
Dr.Zheng's links are also added here for sharing:
http://en.wikipedia.org/wiki/Extract,_transform,_load
http://www.information-management.com/infodirect/2009_109/10014988-1.html
OLAP includes decision support, modeling, information, retrieval, ad hoc reporting/analysis, and what-if analysis. Data Warehouse plays an important role in supporting business analysis applications. It is called a meta-database constructed for quick searching, retrieval, ad hoc queries, and ease of use. The data are normally extracted periodically from an operational database or from a public information service. Data Warehouse is a relational or multidimensional database that may consume hundreds of gigabytes or even terbytes of disk storage. Once the data is loaded into the warehouse, data are accessible via various query and analysis tools that are used for data mining.
The primary reason for using a data warehouse is to optimize business performance. By providing customers and suppliers with the information they need when they need it, the company can improve its relationships and provide better service, e.g. a responsive and efficient supply chain. Hope we will find a good example to explain the importance of Data Warehouse in our presentation.
Implementing a successful data warehouse involves installing a process for gathering data on an ongoing basis organizing it into a meaningful information, and delivering it for evaluation.
Dr.Zheng's links are also added here for sharing:
http://en.wikipedia.org/wiki/Extract,_transform,_load
http://www.information-management.com/infodirect/2009_109/10014988-1.html
Wednesday, September 30, 2009
For the Anti-XML crowd, sometimes you have no choice!
The below link is the National Vulnerability Database XML mapping that major security vendors use, including my old company IBM ISS.
http://nvd.nist.gov/download.cfm#CVE_FEED
Basically, NVD (through a committee decision) has decided to use XML mapping to describe vulnerabilities and the various platforms they affect. Security vendors would then import this information into their databases and map them to their own security vulnerability records.
It's confusing, I know, but the bottom line is sometimes a third party (such as the government in this case) forces companies to read, manipulate, and use XML data. So if you're in the anti-XML camp, sometimes outside influences box you into a corner and you have no real choice. Bummer, eh Phil? :)
http://nvd.nist.gov/download.cfm#CVE_FEED
Basically, NVD (through a committee decision) has decided to use XML mapping to describe vulnerabilities and the various platforms they affect. Security vendors would then import this information into their databases and map them to their own security vulnerability records.
It's confusing, I know, but the bottom line is sometimes a third party (such as the government in this case) forces companies to read, manipulate, and use XML data. So if you're in the anti-XML camp, sometimes outside influences box you into a corner and you have no real choice. Bummer, eh Phil? :)
Sunday, September 27, 2009
Readings on SQL server integration services
Here are two books related to SQL server integration services and they provide comprehensive materials on SQL server integration services. For people interested, you can access to the free book web site above.
1. Microsoft SQL Server 2008 Step by Step by Mike Hotek, Microsoft Press © 2009 (544 pages)
This learn-by-doing tutorial teaches you the essentials for getting started, from data transformation to scripting, to security, and best practices, and so on.
2. SQL Server 2005 Bible by Paul Nielsen
This book includes all the basics, best practices, dozens of targeted examples, and sample code, this comprehensive tutorial is a must for any developer or database administrator transitioning to the new version of SQL Server.
More readings and ideas will be published before our presentation day! Stay tuned.
Friday, September 18, 2009
Handout for presentation - presentation thoughts
Hi everyone,
Dr. Zheng has asked us to publish the little handout that we handed out for our DB middleware presentation last week, and here it is:
http://docs.google.com/View?id=dfpw84nc_7cbr494f3
I think it would be useful if the groups import PPTs into Google Present (part of google docs) and then publish them so everyone can view the presentations before/after the presentations. I think this is useful because sometimes I want to go back and view a point that one of the groups made, but it's impossible if the PPT isn't shared. It just takes 2 minutes to upload and share your PPT or Word file to Google docs, and then publish it and post the link.
While receiving my BS in Technical and Professional Communication many years ago, we learned that often times it is helpful to hand out notes and handouts to strengthen and diversify a PPT presentation. I think Google Present is a good way to avoid having to handout a bunch of paper, but still save your presentation for later use. What do you guys think?
Thanks! -mtm
Dr. Zheng has asked us to publish the little handout that we handed out for our DB middleware presentation last week, and here it is:
http://docs.google.com/View?id=dfpw84nc_7cbr494f3
I think it would be useful if the groups import PPTs into Google Present (part of google docs) and then publish them so everyone can view the presentations before/after the presentations. I think this is useful because sometimes I want to go back and view a point that one of the groups made, but it's impossible if the PPT isn't shared. It just takes 2 minutes to upload and share your PPT or Word file to Google docs, and then publish it and post the link.
While receiving my BS in Technical and Professional Communication many years ago, we learned that often times it is helpful to hand out notes and handouts to strengthen and diversify a PPT presentation. I think Google Present is a good way to avoid having to handout a bunch of paper, but still save your presentation for later use. What do you guys think?
Thanks! -mtm
Sunday, September 13, 2009
MegaHotPot Readings Blog and Abstract for Upcoming Database Middleware Presentation
(Readings Blog and Abstract combined per Dr. Zheng's request - mtm)
Well, the MegaHotPot presentation is coming right up on Tuesday, and we're hammering out the last details. We hope to live up to the great presentations that we've already heard from, and equally hope to set the standard for future presentations.
Well, the MegaHotPot presentation is coming right up on Tuesday, and we're hammering out the last details. We hope to live up to the great presentations that we've already heard from, and equally hope to set the standard for future presentations.
For those who would like to do some readings before the presentation, we have presented six readings in two categories below - 3 from scholarly journals (free PDFs) and 3 casual readings (from well-known web sites).
The scholarly journal readings are quite dense, while the casual readings are very light. Taken together, they give a good overview, good examples, and technical details of database middleware. Give them a once over and see if you agree.
Scholarly Journals:
Transforming Heterogeneous Data with Database Middleware: Beyond Integration
http://dbs.informatik.uni-halle.de/Lehre/Integr/stuff/debull99.pdf
--- For those who want a highly technical read, with a few graphics thrown in for good measure. It's quite a dense read, but it's just six pages so it's not overly exhausting. Good article.
Grid Enabled Relational Database Middleware
http://hep.ac.uk/papers/GGF3Rome2001.pdf
---DataGrids as they relate to DB Middleware discussed here. Also pretty technical, but again not very long. These best part about this article are the different examples they use to show DB middleware in action.
Web Client and ODBC Access to Legacy Database Information: A Low Cost Approach
http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=2233436&blobtype=pdf
--- This is an older article, but it's just five pages and sums up a lot of ODBC/Legacy issues very nicely. Definitely Worth a Read.
Casual Reading:
What Is Middleware? Web site
http://www.middleware.org/whatis.html
--- Great starting place to learn about middleware, and has a nice little section on DB middleware, too.
Open Database Connectivity - Wikipedia
http://en.wikipedia.org/wiki/Open_Database_Connectivity
--- I'm a wikipedia fan, and this nice summary article shows why. Talks about ODBC and how it relates to other DB middleware types. If you don't know much about ODBC, give this a read.
ODBC Microsoft Article
http://support.microsoft.com/kb/110093
--- ODBC talk straight from Microsoft. This reads a little like a sales pitch, but I think it's a good basic coverage on ODBC that takes the above Wiki article and extends even further. The funniest header in this article is "What's So Great About ODBC?"
Classic Microsoft. :)
_____
Abstract of presentation, as presented by Bing:
Abstract of presentaion--Database Middleware
In our presentation, we would start from basic introduction of middleware, different types of middleware,and then dig into our main topic, which is database oriented middleware. we would both provide detailed explanation of different kinds of database middleware and empirical demonstration of one database middleware, which is ODBC.
You would actually learn how database middleware works and differenciate database middleware from other middlewares. With ODBC demo, you would know intuitively how database middleware work and how to connect databases or applications via database middleware.
Our presentation is the second one on week 5. Hope everyone enjoys!
Here is the tentative link through our google presentation: http://docs.google.com/present/edit?id=0Ae7CQJaENDlQZGNuc3g5MndfMjM4Y214Njg3ZjU&hl=en
Classic Microsoft. :)
_____
Abstract of presentation, as presented by Bing:
Abstract of presentaion--Database Middleware
In our presentation, we would start from basic introduction of middleware, different types of middleware,and then dig into our main topic, which is database oriented middleware. we would both provide detailed explanation of different kinds of database middleware and empirical demonstration of one database middleware, which is ODBC.
You would actually learn how database middleware works and differenciate database middleware from other middlewares. With ODBC demo, you would know intuitively how database middleware work and how to connect databases or applications via database middleware.
Our presentation is the second one on week 5. Hope everyone enjoys!
Here is the tentative link through our google presentation: http://docs.google.com/present/edit?id=0Ae7CQJaENDlQZGNuc3g5MndfMjM4Y214Njg3ZjU&hl=en
Subscribe to:
Posts (Atom)
