Archive for July, 2007

As I mentioned in my previous article 10 Ways to Improve System Performance defragmenting your hard drive can speed up file load times resulting in better system performance. One downside of defragmenting your hard drive, however, is the time it can take to run.

In this article I will discuss how to automate defrag using a Windows Command Script and the Windows Task Scheduler. Using these features of Windows we can setup defrag to run at a time when you are not using your computer. You’ll get the benefits of defragging without it having to do it yourself or even having to remember to do it.

Create the Windows Command Script

The first step is to create the Windows Command Script. A command script is simply a text file that contains a sequential list of commands to run - in this case, the defrag command.

To create the command script simply open Notepad. You can find that in the All Programs | Accessories menu under the Start button.

In a new text file you will need the following line:

C:\Windows\System32\defrag C:

That’s it. Save the text file as C:\defrag.cmd. Note that I am not using the standard “txt” extension for a text file, but instead the “cmd” extension that is required for Windows command scripts.

There are two things to consider with this script. First, it assumes that your Windows installation can be found in the C:\Windows folder. If not modify the script to use the folder where you have Windows installed. Second, it assumes that you wish the C drive to be defraged. If not modify the script to target the drive you would like to defrag. For example, if you wish to defrag the D drive instead change the C to a D.

If you would like to review the results of the defrag after it has run then change the script to look like this:

C:\Windows\System32\defrag C: >> C:\defrag.txt

This will cause the results of the defrag to be logged into the file c:\defrag.txt. You can view this file with Notepad to ensure defrag is doing its job.

Creating the Scheduled Task

The second step in automating defrag is to setup a task in the Windows Scheduler. To do so open the Task Scheduler. You can find this in your Windows Control Panel. Once there, some users will need to open the “Performance and Maintenance” category to find the Scheduled Tasks folder.

Click on the icon “Add Scheduled Task”. The Scheduled Task Wizard will appear. Click next.

The next screen asks you to select a program. You should click Browse… and then use the file browser to select the c:\defrag.cmd file that we created in the first step.

Next, select the frequency that you would like the defrag to run. For most users running defrag once per week will yield optimum results. Click next.

Select the time that you would like to run the defrag. You should pick a day and time when you know your PC will be on and idle - that is you won’t be using it. You should also consider other automated tasks you run, such as virus scans, to avoid a conflict. Giving defrag a 1 hour window before other tasks run should be sufficient for most users. Click next.

On some systems you will be prompted to enter and confirm your password. You should enter the name and password that you use to log in to Windows. If the system logs you in automatically, you should be able to leave these fields blank. Click next.

You will be presented with a summary screen. Verify the information is correct and click Finish.

That’s it. You have automated your system to run defrag on a regular schedule and you will experience better system performance as result.

When I bring up the concept of a data warehouse to my clients I often get the same few questions.

• What is a data warehouse?
• Why do I want or need a data warehouse?
• What’s involved in creating a data warehouse?

In this post I will attempt to answer each of these questions.

What is a data warehouse?

First off, do not be overwhelmed by the term “Data Warehouse”. Data Warehouse is just a fancy name for a database with a special purpose. Like other databases, data warehouses are usually setup in Relational Database Management Systems (RDBMS) such as MySQL, MS SQL Server, Oracle or other comparable systems. And like other databases, data warehouses are basically made up of tables, queries and other objects with properties regarding their structure, access rules, etc.

There are two key differences between data warehouses and operational databases, however. First, data warehouses are typically implemented to store a more comprehensive set of your organization’s data, both in terms of operational scope and history. They do this to provide you with a single repository of all operational and historical data. This is the special purpose of a data warehouse.

The second key difference is data warehouse designs are optimized to support reporting and data analysis requirements. Data needs to be retrieved from the system quickly. Operational databases on the other hand are usually optimized to support transaction processing requirements. Data needs to be added and updated quickly.

Why do I want or need a data warehouse?

There are a number of reasons why an organization would implement a data warehouse.

The first reason is to consolidate organizational data into one database to support comprehensive reporting and data analysis. Many organizations use more than one operational system to maintain their data. There are accounting systems, point-of-sale systems, CRM systems, time-clocks, etc, each of which store information in their own way. Within the data stores of these systems is a wealth of information. Unfortunately, many organizations find if very difficult and often impractical to create reports that span these sources in any meaningful way. The data is simply too segregated. By consolidating this information into a data warehouse reporting and data analysis becomes a much easier task. Decision makers and others can be given access to tools that let them see the big picture and drill down on the detail.

The second reason is to maintain a more comprehensive history of information. Some operational systems, in order to maintain adequate performance, need to archive or purge their data after a while. I am currently working with a client who’s point of sale system purges all transactions after 3 months. Because they create 25,000 invoices per month, this makes sense. Allowing that data to accumulate could have adverse effects on performance and cause lag time at the point-of-sale. That would be a customer service no-no, so the point-of-sale system is kept lean and mean. Of course, purging the older data means historical information is lost. By implementing a data warehouse, we are able to preserve and query this historical data indefinitely without degrading the performance of operational systems.

A third reason is to act as a master for data synchronization tools. Data often overlaps in operational systems. Employee information, for example, can be stored in point-of-sale systems, time-clock systems and more. A data warehouse can be a useful tool that helps keep this information in sync. Good syncing tools saves your staff time and help reduce the chance for keying errors. In a similar way, a data warehouse can be used to support other system integration functions like moving transactions from one system to another.

A data warehouse, when implemented properly, is a tool that supports decision making, improves the integrity of your data, and maximizes your staff’s productivity.

What’s involved in creating a data warehouse?

Creating a data warehouse is a process that does require good technical skills and experience in data architecture and software engineering. Nevertheless, the overall process can be understood by most.

The first step in creating a data warehouse is to implement a platform that will support it. This means you will need a server machine running an RDBMS package. Many factors go into selecting the appropriate machine. These include such things as the number of users, the frequency of data migrations, the complexity of reports and data analysis as well as any other applications the server must support. It is important to use some of the proven methods for identifying the hardware requirements of a proposed application server.

As for the RDBMS, I have had great success with MySQL and because it’s open source, the only expense associated with it is the time it takes to set up and administer it. Of course, data warehouses can be implemented with any number of database products. If you already run a specific RDBMS package it will probably be cost effective to use it for your data warehouse as well. It is important to evaluate any proposed package against your specific requirements.

The second step is to design the data model. This involves identifying all of the information that the warehouse must maintain. This is usually done by examining what information is available in your operational systems and identifying specific reports that will be required. From there we create a model - essentially a list of tables - that will hold this data and support the reporting requirements. Once the model is developed it must be implemented on the server. This is done either by writing scripts or using tools provided by the RDBMS vendor.

The third step is to create the data migration tools that will copy data from your operational systems into the data warehouse. In some cases the tools will be scripts or programs that access data using standard interfaces such as ODBC or COM, or a proprietary software development kit (SDK) provided by the vendor.

Other scripts will import data from files in either proprietary formats or common formats like CSV. This will be the case when operational systems don’t offer interfaces and only support export functions or we are receiving files from vendors, customers, field offices or other places.

Many RDBMS tools, including MySQL, have wizards available for creating migration scripts that support common data source formats. This can help speed up the process of creating these tools.

In addition, any number of software developments tools can be used to create the necessary programs.

The fourth step is to setup a migration schedule. To be most useful to decision makers the data warehouse must be updated on a regular basis. Operational data changes constantly and the data warehouse needs to keep pace.

In cases where operational data can be accessed directly, without user intervention, we simply setup the migration tools to run at predetermined times using a task scheduler.

For the cases where data is imported from files, we need to create tools that allow users to initiate the migration by using a File | Open style dialog or by simply uploading, copying or otherwise placing the file into a specified location.

Deciding on the specific migration frequency is simply a balancing act between the need for real time reporting and the load each data migration puts on the data server and other operational systems.

Last but certainly not least, we need to create reports and other data analysis tools that support decision making and other organizational objectives. With a properly designed and implemented data warehouse report writing becomes a breeze. Because the warehouse is built using a SQL-based RDBMS, reports and analysis tools can be developed easily using any number of available tools.

So there you have data warehouses in a nutshell. They help you make more informed decisions, preserve a comprehensive history of your organizations data and improve information management processes. And with the assistance of an experienced data and software consultant building one is relatively straight forward process.

Contact me if you would like to learn more about how to build your data warehouse.

I often find myself working on a client’s machine and thinking, “Boy is this thing slow.” Because I usually work by the hour my clients aren’t always so keen on me spending time outside the scope of my assignment. I try to convince them, however, that by taking some time to improve the system’s performance I can be more productive and, therefore, more cost effective. I believe the same holds true for all computer users. So here are a few tips that can help speed up your PC.

NOTE: Most of these tips apply to Windows XP Users.

1. Eliminate Spyware and Viruses. Spyware and Viruses, i.e. Malware, is often the most significant contributor to performance degradation. They consume system resources and often disable multiple functions. If your machine is infected clean it. Keep your AV and other security tools up to date. Run scans on your machine often. See Securing Your PC for Free for more information.

2. Stop Automatic Loading of Non-Essential Programs. Many software packages like to install a background service that checks for updates. While this is essential for your AV program, you don’t really need it for QuickTime, the Google Toolbar and things like that. These programs do little except hog up your memory and, therefore, negatively affect performance. Many of these background services display themselves as icons in the system tray (next to the clock on your Start Bar), where you can right click on them and close them. However, it is usually best to stop these services from loading in the first place. Some will offer you the option to do that in their preferences or options dialog. Others simply load themselves in various ways with or without your permission.

The first and easiest way to deal with this is the Startup Folder in your Start Menu. If you are not familiar with a program that you find there, then you probably don’t need it. Since these items are just shortcuts, you can delete them safely.

The next place programs are automatically started is the Registry. The Registry is a data file that stores various system and software configurations. If you are familiar with editing the registry using RegEdit then check these keys for startup programs that you might like to eliminate:

• HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Run
• HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run
• HKEY_USERS\.DEFAULT\Software\Microsoft\Windows\CurrentVersion\Run

If you are not familiar with editing the system registry, then you should probably get help from an expert.

The third most common place programs start automatically is via Windows Services. Deciding which services to disable, however, can be a tricky task that most users should avoid. The best way for most users to handle this is by uninstalling programs that you no longer use.

3. Defragment your Hard Drive. The disk defragmentation tool can speed up system performance by reorganizing files for more efficient disk access. The faster files load, the faster your system will be. You can find the defrag tool under the Start | Accessories | System Tools menu or by right clicking on your hard drive in My Computer and selecting Properties | Tools.

4. Clean Up Your Desktop. Your desktop is similar to any other folder on your machine. The main difference is your desktop is always open. Too many files (including shortcuts) and a large background image on your desktop consume memory. By removing the background image and unused icons you can free up some of that precious memory. If you keep documents on your desktop consider moving them to your My Documents folder. This also helps to make backup procedures easier.

5. Eliminate Unused Browser Extensions. If you use too many browser extensions like toolbars, it can negatively affect the speed of your web surfing. Do you really need the Yahoo! Toolbar and the Google Toolbar and the AOL Toolbar? Identify the ones you really need and get rid of the rest.

6. Ensure Your Performance Settings are OK. Right click on My Computer and select Properties. Select the Advanced tab and go to the Performance Settings. Most users will want to “Let Windows choose what’s best …”. If you have special needs you may want to create custom settings. Also, you could do some trial and error here. Change a setting and see how it affects performance. Do this until you find a configuration that works for the way you use your PC.

7. Reduce Multi-Tasking. One of the best things about a multi-tasking operating system is the ability to keep several programs open at once. Unfortunately, it does adversely affect performance. If you don’t need to keep a program open then you should close it.

8. Uninstall Drivers for Devices You No Longer Use. Many devices such as web cams and wireless keyboards require special device drivers to work. These drivers consume memory. If you no longer use a device make sure you uninstall its accompanying driver.

9. Be Patient. When starting a program that seems to take a long time it is usually best to be patient. Have you ever started IE, wait a few seconds, start it again, wait a few more seconds more, start it again and then all of the sudden you have three IE windows open? I assure you that all of that took longer than starting IE once and waiting until it opens.

10. Add Memory. No single upgrade - other than replacing your entire computer - enhances performance like a $100 memory upgrade will. This is especially true for those who multi-task.

It’s hard to determine exactly what type of performance enhancement you will achieve when you apply this advice, but I find that most users do see a difference. If anyone would like to add to these tips, please post your comments. I would like to hear about your experiences with these tips as well.

I have been working with Firefox for over a year now and it has quickly become my favorite browser. I still use Internet Explorer because, as a web developer, I need to ensure that my applications, style sheets and HTML all work properly with what is still the most popular browser. For everyday surfing, however, Firefox is my browser of choice.

One of the key reasons that I like Firefox so much is because of the great extensions available. For developing I love FireBug, FireFTP and the DOM Inspector. They really help in my everyday tasks.

But, what really makes Firefox shine are the extensions that can greatly enhance the safety of your browsing. In my last post I discussed securing your PC using free Anti-Virus, Anti-Spyware and Firewall packages. But there is more to security than just blocking viruses and hackers.

For example, one threat that continues to be pervasive is the phishing scam. The scam usually works something like this. You get an email that looks like it’s from your bank or PayPal or some other financial service. The email notifies you that, for security reasons, you need to login and supply them with some additional information. Of course the email provides you with a handy link so you can jump right there. The link takes you to the con artist’s site where you “login” to the very official looking page. In doing so, you have just given them your id and password, leaving your finances vulnerable.

In addition to some phishing protection built-in to Firefox there is also a great extension to help protect you against this and similar threats. It’s called WOT and it is a web site reputation tool. It allows you to view the ratings other web surfers have given the site you are viewing. WOT quickly notifies you if you are about to visit a phishing site and gives you the option to abandon the site and go elsewhere. Not only does it protect against phishing sites it also warns you about sites that send too much spam or attempt to collect too much personal info. It also integrates with your searches in Yahoo!, Google and others. The appropriate WOT icon displays next to each result giving you a quick visual cue to the safety of the site. WOT already has ratings on over 14 million web sites and as a WOT user you can rate sites too.

Another way threats can manifest themselves on your computer is through the scripts that are embedded into web pages. Most scripts found embedded into web pages are harmless. Typically they simply enhance the web site by providing dynamic content. But scripts can also be used to do malicious things like download spy-ware. NoScript is a Firefox extension that allows you to disable scripts completely. It gives you control over which sites scripts can run - say from trusted sites with applications that require scripting - and which sites are restricted.

These are just two of the great extensions that help make Firefox the safest browser in the world.

Download Firefox here.

Download WOT