Monday, November 10, 2008
How To Block Websties Without Software - Windows
The cool thing is no software needed.
Follow this steps.
Steps:
1] Browse C:\WINDOWS\system32\drivers\etc
2] Find the file named "HOSTS"
3] Open it in notepad
4] Under "127.0.0.1 localhost" Add 127.0.0.2 www.sitenameyouwantblocked.com , and that site will no longer be accessable.
5] Done!
-So-
127.0.0.1 localhost
127.0.0.2 www.blockedsite.com
-->www.blockedsite.com is now unaccessable<--
For every site after that you want to add, just add "1" to the last number in the internal ip (127.0.0.2) and then the addy like before.
IE: 127.0.0.3 www.blablabla.com
127.0.0.4 www.blablabla.com
127.0.0.5 www.blablabla.com
Cracking Zip Password Files with FZC
FZC uses multiple methods of cracking - bruteforce (guessing passwords systematically until the program gets it) or wordlist attacks (otherwise known as dictionary attacks. Instead of just guessing passwords systematically, the program takes passwords out of a "wordlist", which is a text file that contains possible passwords. You can get lots of wordlists at www.theargon.com.).
FZC can be used in order to achieve two different goals: you can either use it to recover a lost zip password which you used to remember but somehow forgot, or to crack zip passwords which you're not supposed to have. So like every tool, this one can be used for good and for evil.
The first thing I want to say is that reading this tutorial... is the easy way to learn how to use this program, but after reading this part of how to use the FZC you should go and check the texts that come with that program and read them all. You are also going to see the phrase "check name.txt" often in this text. These files should be in FZC's directory. They contain more information about FZC.
FZC is a good password recovery tool, because it's very fast and also support resuming so you don't have to keep the computer turned on until you get the password, like it used to be some years ago with older cracking programs. You would probably always get the password unless the password is longer than 32 chars (a char is a character, which can be anything - a number, a lowercase or undercase letter or a symbol such as ! or &) because 32 chars is the maximum value that FZC will accept, but it doesn't really matter, because in order to bruteforce a password with 32 chars you'll need to be at least immortal..heehhe.. to see the time that FZC takes with bruteforce just open the Bforce.txt file, which contains such information.
FZC supports brute-force attacks, as well as wordlist attacks. While brute-force attacks don't require you to have anything, wordlist attacks require you to have wordlists, which you can get from www.theargon.com. There are wordlists in various languages, various topics or just miscellaneous wordlists. The bigger the wordlist is, the more chances you have to crack the password.
We need to keep in mind that are some people might choose some really weird passwords (for example: 'e8t7@$^%*gfh), which are harder to crack and are certainly impossible to crack (unless you have some weird wordlist). If you have a bad luck and you got such a file, having a 200MB list won't help you anymore. Instead, you'll have to use a different type of attack. If you are a person that gives up at the first sign of failure, stop being like that or you won't get anywhere. What you need to do in such a situation is to put aside your sweet xxx MB's list and start using the Brute Force attack.
If you have some sort of a really fast and new computer and you're afraid that you won't be able to use your computer's power to the fullest because the zip cracker doesn't support this kind of technology, it's your lucky day! FZC has multiple settings for all sorts of hardware, and will automatically select the best method.
Now that we've gone through all the theoretical stuff, let's get to the actual commands.
--------------------------------------------------------------------------------
Bruteforce
--------------------------------------------------------------------------------
The command line you'll need to use for using brute force is:
fzc -mb -nzFile.zip -lChr Lenght -cType of chars
Now if you read the bforce.txt that comes with fzc you'll find the description of how works Chr Lenght and the Type of chars, but hey, I'm gonna explain this too. Why not, right?... (but remember look at the bforce.txt too)
For Chr Lenght you can use 4 kind of switches...
-> You can use range -> 4-6 :it would brute force from 4 Chr passwors to 6 chr passwords
-> You can use just one lenght -> 5 :it would just brute force using passwords with 5 chars
-> You can use also the all number -> 0 :it would start brute forcing from passwords with lenght 0 to lenght 32, even if you are crazy i don't think that you would do this.... if you are thinking in doing this get a live...
-> You can use the + sign with a number -> 3+ :in this case it would brute force from passwords with lenght 3 to passwords with 32 chars of lenght, almost like the last option...
For the Type of chars we have 5 switches they are:
-> a for using lowercase letters
-> A for using uppercase letters
-> ! for using simbols (check the Bforce.txt if you want to see what simbols)
-> s for using space
-> 1 for using numbers
Example:
If you want to find a password with lowercase and numbers by brute force you would just do something like:
fzc -mb -nzTest.zip -l4-7 -ca1
This would try all combinations from passwords with 4 chars of lenght till 7 chars, but just using numbers and lowercase.
*****
hint
*****
You should never start the first brute force attack to a file using all the chars switches, first just try lowercase, then uppercase, then uppercase with number then lowercase with numbers, just do like this because you can get lucky and find the password much faster, if this doesn't work just prepare your brain and start with a brute force that would take a lot of time. With a combination like lowercase, uppercase, special chars and numbers.
--------------------------------------------------------------------------------
Wordlis
--------------------------------------------------------------------------------
Like I said in the bottom and like you should be thinking now, the wordlist is the most powerfull mode in this program. Using this mode, you can choose between 3 modes, where each one do some changes to the text that is in the wordlist, I'm not going to say what each mode does to the words, for knowing that just check the file wlist.txt, the only thing I'm going to tell you is that the best mode to get passwords is mode 3, but it takes longer time too.
To start a wordlist attak you'll do something like.
fzc -mwMode number -nzFile.zip -nwWordlist
Where:
Mode number is 1, 2 or 3 just check wlist.txt to see the changes in each mode.
File.zip is the filename and Wordlist is the name of the wordlist that you want to use. Remember that if the file or the wordlist isn't in the same directory of FZC you'll need to give the all path.
You can add other switches to that line like -fLine where you define in which line will FZC start reading, and the -lChar Length where it will just be read the words in that char length, the switche works like in bruteforce mode.
So if you something like
fzc -mw1 -nztest.zip -nwMywordlist.txt -f50 -l9+
FZC would just start reading at line 50 and would just read with length >= to 9.
Example:
If you want to crack a file called myfile.zip using the "theargonlistserver1.txt" wordlist, selecting mode 3, and you wanted FZC to start reading at line 50 you would do:
fzc -mw3 -nzmyfile.zip -nwtheargonlistserver1.txt -f50
--------------------------------------------------------------------------------
Resuming
--------------------------------------------------------------------------------
Other good feature in FZC is that FZC supports resuming. If you need to shutdown your computer and FZC is running you just need to press the ESC key, and fzc will stop. Now if you are using a brute force attack the current status will be saved in a file called resume.fzc but if you are using a wordlist it will say to you in what line it ended (you can find the line in the file fzc.log too).
To resume the bruteforce attack you just need to do:
fzc -mr
And the bruteforce attack will start from the place where it stopped when you pressed the ESC key.
But if you want to resume a wordlist attack you'll need to start a new wordlist attack, saying where it's gonna start. So if you ended the attack to the file.zip in line 100 using wordlist.txt in mode 3 to resume you'll type
fzc -mw3 -nzfile.zip -nwwordlist.txt -f100
Doing this FZC would start in line 100, since the others 99 lines where already checked in an earlier FZC session.
Anonymity of Proxy
What your browser transmits to a web-server:
a name and a version of an operating system
a name and a version of a browser
configuration of a browser (display resolution, color depth, java / javascript support, ...)
IP-address of a client
Other information
The most important part of such information (and absolutely needless for a web-server) is information about IP-address. Using your IP it is possible to know about you the following:
a country where you are from
a city
your provider?s name and e-mail
your physical address
Information, transmitted by a client to a server is available (accessible) for a server as environment variables. Every information unit is a value of some variable. If any information unit is not transmitted, then corresponding variable will be empty (its value will be undetermined).
These are some environment variables:
REMOTE_ADDR ? IP address of a client
HTTP_VIA ? if it is not empty, then a proxy is used. Value is an address (or several addresses) of a proxy server, this variable is added by a proxy server itself if you use one.
HTTP_X_FORWARDED_FOR ? if it is not empty, then a proxy is used. Value is a real IP address of a client (your IP), this variable is also added by a proxy server if you use one.
HTTP_ACCEPT_LANGUAGE ? what language is used in browser (what language a page should be displayed in)
HTTP_USER_AGENT ? so called "a user?s agent". For all browsers this is Mozilla. Furthermore, browser?s name and version (e.g. MSIE 5.5) and an operating system (e.g. Windows 98) is also mentioned here.
HTTP_HOST ? is a web server?s name
This is a small part of environment variables. In fact there are much more of them (DOCUMENT_ROOT, HTTP_ACCEPT_ENCODING, HTTP_CACHE_CONTROL, HTTP_CONNECTION, SERVER_ADDR, SERVER_SOFTWARE, SERVER_PROTOCOL, ...). Their quantity can depend on settings of both a server and a client.
These are examples of variable values:
REMOTE_ADDR = 194.85.1.1
HTTP_ACCEPT_LANGUAGE = ru
HTTP_USER_AGENT = Mozilla/4.0 (compatible; MSIE 5.0; Windows 98)
HTTP_HOST = www.webserver.ru
HTTP_VIA = 194.85.1.1 (Squid/2.4.STABLE7)
HTTP_X_FORWARDED_FOR = 194.115.5.5
Anonymity at work in Internet is determined by what environment variables "hide" from a web-server.
If a proxy server is not used, then environment variables look in the following way:
REMOTE_ADDR = your IP
HTTP_VIA = not determined
HTTP_X_FORWARDED_FOR = not determined
According to how environment variables "hided" by proxy servers, there are several types of proxies
Transparent Proxies
They do not hide information about your IP address:
REMOTE_ADDR = proxy IP
HTTP_VIA = proxy IP
HTTP_X_FORWARDED_FOR = your IP
The function of such proxy servers is not the improvement of your anonymity in Internet. Their purpose is information cashing, organization of joint access to Internet of several computers, etc.
Anonymous Proxies
All proxy servers, that hide a client?s IP address in any way are called anonymous proxies
Simple Anonymous Proxies
These proxy servers do not hide a fact that a proxy is used, however they replace your IP with its own:
REMOTE_ADDR = proxy IP
HTTP_VIA = proxy IP
HTTP_X_FORWARDED_FOR = proxy IP
These proxies are the most widespread among other anonymous proxy servers.
Distorting Proxies
As well as simple anonymous proxy servers these proxies do not hide the fact that a proxy server is used. However a client?s IP address (your IP address) is replaced with another (arbitrary, random) IP:
REMOTE_ADDR = proxy IP
HTTP_VIA = proxy IP
HTTP_X_FORWARDED_FOR = random IP address
High Anonymity Proxies
These proxy servers are also called "high anonymity proxy". In contrast to other types of anonymity proxy servers they hide a fact of using a proxy:
REMOTE_ADDR = proxy IP
HTTP_VIA = not determined
HTTP_X_FORWARDED_FOR = not determined
That means that values of variables are the same as if proxy is not used, with the exception of one very important thing ? proxy IP is used instead of your IP address.
Summary
Depending on purposes there are transparent and anonymity proxies. However, remember, using proxy servers you hide only your IP from a web-server, but other information (about browser configuration) is accessible!
How To Remove and Add Right-Click Menu Items from Files and Folders
A lot of programs you install will add themselves to the right-click menu of your files and/or folders. And most times, you have no choice in the matter and, as a result, your right-click menu can get very long with added items you don't even use. The last person I was helping with this had a right context menu so long that the Rename option was no longer visible!
Fortunately, you can easily remove those unwanted menu items, if you know the registry values to edit. And it's not at all difficult once you know the keys responsible for the additions.
For Files, the secret lies in the "context menu handlers" under the shellex subkey for "All Files" which, in the registry, is nothing but an asterisk - like a dos wildcard, which means the values entered apply to all files. It is at the very top of the Root key, right here:
HKEY_CLASSES_ROOT\*\shellex\ContextMenuHandlers
Click the the + sign next to the ContextMenuHandlers key, to expand it.
Now you will see some of the programs that have added items to your right-click menu. Simply delete the program keys you don't want.
Yup! It's that simple. If deleting makes you uneasy, just export the key before deleting it. Or, instead of deleting the values, disable them. Simply double click the default value for the program on the right hand pane and rename the clsid value by placing a period or dash in front of it.
ie; - {b5eedee0-c06e-11cf-8c56-444553540000}
Then exit the registry, refresh, and right click a file to see if the item was removed from the menu.
Some programs - like WinZip or WinRar - will add several items to your right click menu but all of them will be removed by deleting or disabling their one context menu handler.
Note that the above key only applies to the right click menu of files.
To remove entries from the right click context menu of folders, you need to navigate to the Folder and Drive keys:
HKEY_CLASSES_ROOT\Folder\shellex\ContextMenuHandlers
HKEY_CLASSES_ROOT\Drive\shellex\ContextMenuHandlers
All you have to do is follow the same procedure as for Files - either disable or delete items you wish to remove.
Adding Items
Adding Items to the right click menu of Files and Folders is also fairly simple using the Registry. It just involves the creation of a few new keys for each item you wish to add. You edit the same keys used for removing items. Let's use Notepad as an example of an item you'd like to add to the right click menu of all your files or folders.
For folders, go to this key:
HKEY_CLASSES_ROOT\Folder
Click the + sign next to Folder and expand it so that the Shell key is visible. Right click the Shell key and choose New>Key and name the key Notepad or whatever else you'd prefer (whatever the key is named is what will appear in the right-click menu). Now right click the new key you made and create another key named Command. Then, in the right hand pane, double click "Default" and enter Notepad.exe as the value.
Exit the registry, refresh, and right click any folder. Notepad should now be on the context menu.
For files, go here again:
HKEY_CLASSES_ROOT\*
Expand the * key and see if a Shell key exists. If it does exist, follow the same procedure as for folders. If it does not exist, you'll have to create a new Shell first. Just right click the * key and choose New>Key and name it Shell. Then right click the Shell key and continue on the same way you did for adding items to the right click menu of folders.
Once done, Notepad should appear as an option in the right click menu of all your files.
How to Customize your Start button - Windows
After you download it, follow the guide here:
Step 1:
A - Run "Resource Hacker" and open the file "%windir%\Explorer.exe".
B - You see a Tree of all Resources in this file, expand the "String Table"
C - Find the "start" and replace it with your own text. then press the [Compile Script] button.
D - Save "Explorer.exe" as "MyStart.exe" DONT save it as Explorer.exe, do "save as.." and give it a new name.
E - Quit "Resource Hacker".
Step 2:
A - Click on the [Start] button and choose the "Run..." item from the start menu. (Or use the shortcut key WinKey+R)
B - Type "RegEdit" in the Run "Dialog Box". And then press the [Ok] buton to run the "Registry Editor" program.
C - Go to: "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WindowsNT\CurrentVersion\Winlogon" and find the "Shell" property.
D - Replace value of the "Shell" property to "MyStart.exe".
E - Quit "Registry Editor".
F - Restart your system.
Note about Registry Editor:
if you did not find the key "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WindowsNT\CurrentVersion\Winlogon", you can search the Registry for the "Explorer.exe", to do this use the Edit Menu | Find Next (Ctrl+F).
10 reasons why PCs crash
You have just been struck by the Blue Screen of Death. Anyone who uses Mcft Windows will be familiar with this. What can you do? More importantly, how can you prevent it happening?
1 Hardware conflict
The number one reason why Windows crashes is hardware conflict. Each hardware device communicates to other devices through an interrupt request channel (IRQ). These are supposed to be unique for each device.
For example, a printer usually connects internally on IRQ 7. The keyboard usually uses IRQ 1 and the floppy disk drive IRQ 6. Each device will try to hog a single IRQ for itself.
If there are a lot of devices, or if they are not installed properly, two of them may end up sharing the same IRQ number. When the user tries to use both devices at the same time, a crash can happen. The way to check if your computer has a hardware conflict is through the following route:
* Start-Settings-Control Panel-System-Device Manager.
Often if a device has a problem a yellow '!' appears next to its description in the Device Manager. Highlight Computer (in the Device Manager) and press Properties to see the IRQ numbers used by your computer. If the IRQ number appears twice, two devices may be using it.
Sometimes a device might share an IRQ with something described as 'IRQ holder for PCI steering'. This can be ignored. The best way to fix this problem is to remove the problem device and reinstall it.
Sometimes you may have to find more recent drivers on the internet to make the device function properly. A good resource is www.driverguide.com. If the device is a soundcard, or a modem, it can often be fixed by moving it to a different slot on the motherboard (be careful about opening your computer, as you may void the warranty).
When working inside a computer you should switch it off, unplug the mains lead and touch an unpainted metal surface to discharge any static electricity.
To be fair to Mcft, the problem with IRQ numbers is not of its making. It is a legacy problem going back to the first PC designs using the IBM 8086 chip. Initially there were only eight IRQs. Today there are 16 IRQs in a PC. It is easy to run out of them. There are plans to increase the number of IRQs in future designs.
2 Bad Ram
Ram (random-access memory) problems might bring on the blue screen of death with a message saying Fatal Exception Error. A fatal error indicates a serious hardware problem. Sometimes it may mean a part is damaged and will need replacing.
But a fatal error caused by Ram might be caused by a mismatch of chips. For example, mixing 70-nanosecond (70ns) Ram with 60ns Ram will usually force the computer to run all the Ram at the slower speed. This will often crash the machine if the Ram is overworked.
One way around this problem is to enter the BIOS settings and increase the wait state of the Ram. This can make it more stable. Another way to troubleshoot a suspected Ram problem is to rearrange the Ram chips on the motherboard, or take some of them out. Then try to repeat the circumstances that caused the crash. When handling Ram try not to touch the gold connections, as they can be easily damaged.
Parity error messages also refer to Ram. Modern Ram chips are either parity (ECC) or non parity (non-ECC). It is best not to mix the two types, as this can be a cause of trouble.
EMM386 error messages refer to memory problems but may not be connected to bad Ram. This may be due to free memory problems often linked to old Dos-based programmes.
3 BIOS settings
Every motherboard is supplied with a range of chipset settings that are decided in the factory. A common way to access these settings is to press the F2 or delete button during the first few seconds of a boot-up.
Once inside the BIOS, great care should be taken. It is a good idea to write down on a piece of paper all the settings that appear on the screen. That way, if you change something and the computer becomes more unstable, you will know what settings to revert to.
A common BIOS error concerns the CAS latency. This refers to the Ram. Older EDO (extended data out) Ram has a CAS latency of 3. Newer SDRam has a CAS latency of 2. Setting the wrong figure can cause the Ram to lock up and freeze the computer's display.
Mcft Windows is better at allocating IRQ numbers than any BIOS. If possible set the IRQ numbers to Auto in the BIOS. This will allow Windows to allocate the IRQ numbers (make sure the BIOS setting for Plug and Play OS is switched to 'yes' to allow Windows to do this.).
4 Hard disk drives
After a few weeks, the information on a hard disk drive starts to become piecemeal or fragmented. It is a good idea to defragment the hard disk every week or so, to prevent the disk from causing a screen freeze. Go to
* Start-Programs-Accessories-System Tools-Disk Defragmenter
This will start the procedure. You will be unable to write data to the hard drive (to save it) while the disk is defragmenting, so it is a good idea to schedule the procedure for a period of inactivity using the Task Scheduler.
The Task Scheduler should be one of the small icons on the bottom right of the Windows opening page (the desktop).
Some lockups and screen freezes caused by hard disk problems can be solved by reducing the read-ahead optimisation. This can be adjusted by going to
* Start-Settings-Control Panel-System Icon-Performance-File System-Hard Disk.
Hard disks will slow down and crash if they are too full. Do some housekeeping on your hard drive every few months and free some space on it. Open the Windows folder on the C drive and find the Temporary Internet Files folder. Deleting the contents (not the folder) can free a lot of space.
Empty the Recycle Bin every week to free more space. Hard disk drives should be scanned every week for errors or bad sectors. Go to
* Start-Programs-Accessories-System Tools-ScanDisk
Otherwise assign the Task Scheduler to perform this operation at night when the computer is not in use.
5 Fatal OE exceptions and VXD errors
Fatal OE exception errors and VXD errors are often caused by video card problems.
These can often be resolved easily by reducing the resolution of the video display. Go to
* Start-Settings-Control Panel-Display-Settings
Here you should slide the screen area bar to the left. Take a look at the colour settings on the left of that window. For most desktops, high colour 16-bit depth is adequate.
If the screen freezes or you experience system lockups it might be due to the video card. Make sure it does not have a hardware conflict. Go to
* Start-Settings-Control Panel-System-Device Manager
Here, select the + beside Display Adapter. A line of text describing your video card should appear. Select it (make it blue) and press properties. Then select Resources and select each line in the window. Look for a message that says No Conflicts.
If you have video card hardware conflict, you will see it here. Be careful at this point and make a note of everything you do in case you make things worse.
The way to resolve a hardware conflict is to uncheck the Use Automatic Settings box and hit the Change Settings button. You are searching for a setting that will display a No Conflicts message.
Another useful way to resolve video problems is to go to
* Start-Settings-Control Panel-System-Performance-Graphics
Here you should move the Hardware Acceleration slider to the left. As ever, the most common cause of problems relating to graphics cards is old or faulty drivers (a driver is a small piece of software used by a computer to communicate with a device).
Look up your video card's manufacturer on the internet and search for the most recent drivers for it.
6 Viruses
Often the first sign of a virus infection is instability. Some viruses erase the boot sector of a hard drive, making it impossible to start. This is why it is a good idea to create a Windows start-up disk. Go to
* Start-Settings-Control Panel-Add/Remove Programs
Here, look for the Start Up Disk tab. Virus protection requires constant vigilance.
A virus scanner requires a list of virus signatures in order to be able to identify viruses. These signatures are stored in a DAT file. DAT files should be updated weekly from the website of your antivirus software manufacturer.
An excellent antivirus programme is McAfee VirusScan by Network Associates ( www.nai.com). Another is Norton AntiVirus 2000, made by Symantec ( www.symantec.com).
7 Printers
The action of sending a document to print creates a bigger file, often called a postscript file.
Printers have only a small amount of memory, called a buffer. This can be easily overloaded. Printing a document also uses a considerable amount of CPU power. This will also slow down the computer's performance.
If the printer is trying to print unusual characters, these might not be recognised, and can crash the computer. Sometimes printers will not recover from a crash because of confusion in the buffer. A good way to clear the buffer is to unplug the printer for ten seconds. Booting up from a powerless state, also called a cold boot, will restore the printer's default settings and you may be able to carry on.
8 Software
A common cause of computer crash is faulty or badly-installed software. Often the problem can be cured by uninstalling the software and then reinstalling it. Use Norton Uninstall or Uninstall Shield to remove an application from your system properly. This will also remove references to the programme in the System Registry and leaves the way clear for a completely fresh copy.
The System Registry can be corrupted by old references to obsolete software that you thought was uninstalled. Use Reg Cleaner by Jouni Vuorio to clean up the System Registry and remove obsolete entries. It works on Windows 95, Windows 98, Windows 98 SE (Second Edition), Windows Millennium Edition (ME), NT4 and Windows 2000.
Read the instructions and use it carefully so you don't do permanent damage to the Registry. If the Registry is damaged you will have to reinstall your operating system. Reg Cleaner can be obtained from www.jv16.org
Often a Windows problem can be resolved by entering Safe Mode. This can be done during start-up. When you see the message "Starting Windows" press F4. This should take you into Safe Mode.
Safe Mode loads a minimum of drivers. It allows you to find and fix problems that prevent Windows from loading properly.
Sometimes installing Windows is difficult because of unsuitable BIOS settings. If you keep getting SUWIN error messages (Windows setup) during the Windows installation, then try entering the BIOS and disabling the CPU internal cache. Try to disable the Level 2 (L2) cache if that doesn't work.
Remember to restore all the BIOS settings back to their former settings following installation.
9 Overheating
Central processing units (CPUs) are usually equipped with fans to keep them cool. If the fan fails or if the CPU gets old it may start to overheat and generate a particular kind of error called a kernel error. This is a common problem in chips that have been overclocked to operate at higher speeds than they are supposed to.
One remedy is to get a bigger better fan and install it on top of the CPU. Specialist cooling fans/heatsinks are available from www.computernerd.com or www.coolit.com
CPU problems can often be fixed by disabling the CPU internal cache in the BIOS. This will make the machine run more slowly, but it should also be more stable.
10 Power supply problems
With all the new construction going on around the country the steady supply of electricity has become disrupted. A power surge or spike can crash a computer as easily as a power cut.
If this has become a nuisance for you then consider buying a uninterrupted power supply (UPS). This will give you a clean power supply when there is electricity, and it will give you a few minutes to perform a controlled shutdown in case of a power cut.
It is a good investment if your data are critical, because a power cut will cause any unsaved data to be lost.
Sunday, November 9, 2008
What is MUMPS Programming +a disease or what :)
Extreme Database programming with MUMPS Globals
Globals: An alternative to the relational view
The really important heart of MUMPS is its data storage mechanism. This is based on what are known as Global Variables, more commonly known simply as Globals. Globals are an incredibly simple concept, and yet incredibly powerful.
Many people get turned off by Globals. They are a primitive structure. There are none of the controls, safety nets or value-added mechanisms that “proper” databases provide. As a result, MUMPS is often dismissed as irrelevant, insufficient or somehow, just plain wrong. To do so is to ignore a data storage mechanism that is lean, mean and totally malleable. In the right hands, that lack of baggage and overhead can be an incredibly liberating experience. In the wrong hands it can be a recipe for disaster. It’s a bit like a dangerous, extreme sport such as free-form mountain climbing. Few “proper” sportspeople will recommend it, but it’s the fastest, most efficient and most exhilarating way up that mountain if you can master the skills to climb without the safety-related paraphernalia. The climbing equipment companies won’t recommend it either, because it questions the need for their products!
So if you think you’re up to the challenge, and ready to accept that there may be another way to consider data storage than the entrenched relational and SQL view of the world, let’s dig deeper.
All MUMPS systems and their modern derivatives, such as Caché, use Globals as the basis of their storage mechanism. Many modern MUMPS systems and/or value-added packages layer more “conventional” views onto the core Global construct, eg a global-based system can be layered to present a logical Object database, relational database or native XML database characteristics. Indeed the same physical Global-based database could potentially be logically presented and viewed in more than one, if not all, of these ways. As a result, there are growing numbers of developers who are unaware that Globals are there under the covers, and unaware of what they are, how they work and how you can use them. This paper will help you discover this hidden world.
So what are Globals?
Put simply, a Global is a persistent, sparse, multi-dimensional array, containing a text value. Actually MUMPS allows the use of both persistent and non-persistent multi-dimensional arrays, the latter known as “local arrays”.
MUMPS allows the use of both numeric and textual subscripting. So in MUMPS, you can have an array such as:
Employee(company,country,office,employeeNumber) = employeeDetails
eg
Employee(“MGW”,”
In this example, the data items that make up the employee details (name, position, telephone number) have been appended together with the back-apostrophe character as a delimiter. MUMPS does not impose any controls or rules over how you construct your data structures : there’s no schema or data dictionary for describing your database records. This gives you incredible flexibility and speed of development. You can arbitrarily assign one or more data delimiters to break the text value of an array record into a number of “fields”. The total string length that can be assigned to a single array record depends on the MUMPS implementation, but in Caché it is up to 32k. The string length that is actually stored is variable, and you can see that by using a delimiter character, individual fields are variable length also. This makes MUMPS globals a very efficient data storage mechanism : there’s almost no disc space holding empty, unused spaces.
Now in the example above, the employee record will be held in what is known as a “Local Array”. If you were to exit from your MUMPS session, the array would disappear, just like a PHP array once the page or session has gone.
Now here’s the fun bit. To store this employee record permanently to disc, ie as a Global, just add a “^” in front of the array name:
^Employee(company,country,office,employeeNumber) = employeeDetails
eg
^Employee(“MGW”,”
That’s all there is to it!
So, to create such a global node, you would use the MUMPS command “set”, ie:
set ^Employee(“MGW”,”
Something that confuses and frightens people about MUMPS is that most commands can be abbreviated down to a single letter (in upper or lower case), so you’ll often see the following instead:
s ^Employee(“MGW”,”
Now when you exit from MUMPS, the record will persist on disc permanently. When you come back at any time in the future, you can retrieve the record straight off the disc by using the global reference:
^Employee(“MGW”,”
To recover a global node within your programs, you typically use the “set” command to assign the value held in a global record (or “node”) to a local variable, eg:
Set companyRecord= ^Employee(“MGW”,”
The variable companyRecord will now contain the string value:
“Rob Tweed`Director`020 8404 3054”
Globals can have as many subscripts as you like, and the subscripts can be any mixture of text and numbers (real or integers). String subscripts must be surrounded by double quotes, while numbers are not.
Most MUMPS implementations have practical limits over the total subscript length, so you can’t go totally overboard, but you’ll find that it will cater for all of your requirements.
To delete a global node, you use the “kill” command, eg:
Kill ^Employee(“MGW”,”
That’s literally all there is to Globals. The real trick is how to make such a primitive data structure work for you. That’s what the rest of this paper will do. We’ll attempt to do this in such a way that the relational database programmer can understand the equivalent techniques and representations that a MUMPS database practitioner would use.
Remember that there’s nothing in MUMPS that will enforce a particular database design methodology, and its up to you to add the controls and checks that will ensure your database is logically consistent and error-free. That means you’ll be doing a lot of work that a more conventional DBMS would otherwise do for you, but you’ll soon find that you can automate the most repetitive tasks, and make light work of the management of a MUMPS-based database.
Creating a simple multi-level hierarchy
You may have multiple levels of subscripting simultaneously in your globals, eg:
^Employee(“MGW”,”
^Employee(“MGW”,”
^Employee(“MGW”,”
Here we’re specifying the number of employees at a given office at the third level of subscripting, and the actual employee record at the fourth level.
Links between different globals are for the programmer to define. Once again MUMPS provides no automatic indexing or cross-linking itself.
Hence, we could have a telephone number global that acts as an index to the employee global, eg:
^EmployeeTelephone(“020 8404 3054) = “MGW`UK`London`1”
^EmployeeTelephone(“01737 371457”) = “MGW`UK`London`2”
In this example, the data value stored against each telephone number holds the subscripts for the associated employee record, concatenated together. By knowing the telephone number, all we’d have to do is break the data value apart using the back apostrophe as a delimiter, and we could retrieve the main employee record.
For example:
S telNo=”020 8404 3054”
S indexData=^EmployeeTelephone(telNo)
S company=$piece(indexData,”`”,1)
S country=$piece(indexData,”`”,2)
S office=$piece(indexData,”`”,3)
S employeeNo=$piece(indexData,”`”,4)
S record=^Employee(company,country,office,employeeNo)
S employee=$piece(record,”`”,1)
Note the use of the $piece MUMPS function to separate out the individual “pieces” in the index data string and the employee record string.
One of the great things is that nothing in MUMPS has to be pre-declared. You decide when and how to create, modify or delete global records – it’s all automatically and dynamically handled for you. You can add further pieces to your global at any time without any need for declarations whatsoever.
Setting, Getting and Deleting Globals
To summarise, in MUMPS, globals are created and retrieved using the “Set” command, and deleted using the “Kill” command.
1) Creating a global record:
Set ^Employee(“MGW”,”
This creates the global reference, saving the record to disc.
2) Retrieving a global record
Set data=^Employee(“MGW”,”
This retrieves the specified global and places the data value into a local variable named “data”.
3) Deleting a global record:
kill ^Employee(“MGW”,”
This permanently and immediately deletes the specified global record from disc. Be very careful with the Kill command – it’s both extremely simple to use and incredibly dangerous. If you specify fewer subscripts, all lower-level subscripts will be deleted. If you specify no subscripts at all, the entire global will be deleted, eg:
Kill ^Employee(“MGW”,”
This will delete all records for all
Kill ^Employee
This will delete the entire ^Employee global !
Traversing a Global
One of the most frequent things you need to do is traverse some or all of a global. For example, let’s say you want to manipulate all the employee records, perhaps to display a list so that the user can select one of them, or to count them. To do this, you use the $order MUMPS function.
This operates on a single subscript level, and returns the next subscript value in collating sequence that exists at that level in the global. You can specify some starting value, and the $order function will find the next value that exists in collating sequence. To find the first subscript at the specified level, use a starting value of null (“”)
So, to find the first company in our Employee global :
S company=””
S company=$order(^Employee(company))
When the last value is found, the next time the $order function is run, it will return a null value. So, to get all companies in the Employee global, we would do the following:
S company=””
For s company=$order(^Employee(company)) quit:company=”” do
. ; do something with the company
This demonstrates some of the interesting features of the brevity of MUMPS coding:
The For command followed by two spaces sets up an infinite loop.
- The Quit:company=”” provides the termination condition for the loop, and uses a construct known as the “post-conditional”. What this construct is saying is “if the value of company is null, then quit the For loop”. The Quit must be followed by two spaces if any other command is written after it.
- The “do” at the end of the line means execute an code that follows at the next “dot” level. The “do” will get executed on every iteration of the For loop where company is non-null
- The code that is to be executed on every loop is written after a single dot. In essence, any lines starting with a dot constitute a subroutine that is executed by that “do” at the end of the second line in the example.
So, we seed the $order with a null to make sure that it starts by finding the first subscript value that is stored in the global. We loop through each value until we exhaust the values that are saved, in which case we get a null value returned by the $order. We then detect this null value and terminate the loop.
We can extend this to traverse the entire global. To do this we loop through each subscript level, starting at the first, and progressively moving to the next one:
S company=””
For s company=$order(^Employee(company)) quit:company=”” do
. s country=””
. For s country=$order(^Employee(company,country)) quit:country=”” do
. . s office=””
. . For s office=$order(^Employee(company,country,office)) quit:office=”” do
. . . s employeeNo=””
. . . For s employeeNo=$order(^Employee(company,country,office,employeeNo))
quit:employeeNo=”” do
. . . . s record=^Employee(company,country,office,employeeNo)
. . . . ; do something with the record
Note here the way we’ve nested the “dot” levels to create a hierarchy of nested subroutines.
Checking to see if a Global node exists
You’ll often want to know if a particular global node exists. You can use the MUMPS $data function for this, eg:
if $data(^Employee(company)) do xxx
Read this as “if ^Employee(company) exists, then execute the xxx subroutine. The $data function can be abbreviated to $d.
$data will actually return a number of different values.
If data exists at the specified level of subscripting, and there are no sub-nodes, a value of 1 is returned
- If data exists at the specified level of subscripting, and there are sub-nodes, a value of 11 is returned
- If no data exists at the specified level of subscripting, but there are sub-nodes, a value of 10 is returned
- If no data and no subnodes exist at the specified level of subscripting, a value of 0 is returned.
In MUMPS, any non-zero value is evaluated as “true” when a logical operator such as if is used. Hence, the first three values returned by $data (ie 1, 10 and 11) will evaluate as true. The last situation (no data and no subnodes) will evaluate as false.
For example, consider the following global:
^test=3
^test(“a”)=1
^test(“a”,”b”,”c”)=2
^test(“a”,”b”,”d”)=3
$data(^test) = 11
$data(^test(“a”,”b”)=10
$data(^test(“a”,”b”,”c”)=1
$data(^test(“b”)=0
Preventing Undefined errors
If you try to retrieve a global node that doesn’t exist, MUMPS will produce a run-time error, eg
So, based on the example we used in the previous section on $data:
$get(^test) = 3
$get(^test(“a”,”b”)=”” ; because no data exists at this level of subscripting
$get(^test(“a”,”b”,”c”)=1
$get(^test(“b”)=””
$get(^nonExistentGlobal)=””
Examining Globals
Caché and all other MUMPS systems include utilities for examining globals. The simplest, command line utility is ^%G which you’ll find in Caché and some other MUMPS implementations. Run this from a terminal session:
USER> D ^%G
Global ^ Employee
Typing just the Global name will list the entire global. However, you can restrict the listing to specific subscripts, eg:
Employee()
This will list all values for the first subscript only
Employee(“MGW”)
This will list all Employee records with a first subscript value of MGW
Employee(“MGW”,)
This will list all second subscripts for Employee records with a first subscript of “MGW”
Alternatively, Caché provides a graphical user interface tool called Caché Explorer that allows you to view and edit globals.
Conclusions
We’ve now covered the core basics of what globals are and how they can be created and manipulated. The next chapter will look at globals from the perspective of someone familiar with a relational database.
You’ve probably already realised that MUMPS globals impose very few controls or limitations over what you do. That’s both a great thing – it allows you to very rapidly and flexibly design, implement and redesign your database structures – and a dangerous thing – in the wrong hands it can be a recipe for a completely uncontrolled mess. MUMPS leaves the discipline to you, the programmer. There are no safety nets, but on the other hand, there’s almost no limit to what you can achieve or how you achieve it. You’ll find that the efficiency of coding and execution is what really makes MUMPS an exciting and exhilarating environment to work in.
Once you try using MUMPS globals persistent storage, you’ll probably begin to wonder why all databases couldn’t work this way! Its simple, intuitive, flexible and the performance outstrips any relational database. Its also available for pretty much every platform, and will scale to enormous systems – some of the biggest interactive systems in the world run in MUMPS, some with tens of thousands of concurrent users.
However, if you think you need those controls and safety nets and the features that the relational world believe are essential “must-have” features, then MUMPS is definitely not for you. If you’re still determined to go free-form mountain climbing, move on to the next chapter !
RDBMS/SQL vs. MUMPS
This chapter will set out the salient differences between a standard Relational Database Management Systems (RDBMS) managed through SQL and the MUMPS data repository.
Defining the data
Let us start with the basics – defining the data
As an example, we shall use a simple database consisting of three tables:
1. A table of customers (CUSTOMER).
2. A table of orders (ORDER).
3. A table indicating the items making up an individual order (ITEM).
CUSTOMER custNo name address totalOrders
|
|
+------< ORDER orderNo custNo orderDate invoiceDate totalValue
|
|
+------< ITEM orderNo itemNo price
Table names are shown in bold. The primary key of each table is shown underlined.
CUSTOMER
custNo The Customer’s (unique) number.
name The Customer’s name.
address The Customer’s address.
totalOrders The number of orders placed by the customer to date.
ORDER
orderNo The order number.
custNo The relevant customer number (a foreign key from CUSTOMER).
orderDate The Date of the Order.
invoiceDate The Date of the Invoice.
totalValue The Value of the order.
ITEM
orderNo The order number (corresponding key from ORDER).
itemNo The item (or part) number.
price The price of the item to the customer (including any discount).
The one-to-many relationships are shown in the diagram. Each customer can have many orders and each order can be made up of any number of items (or parts).
The number of orders for a particular customer (CUSTOMER.totalOrders) is the total number of orders placed by the customer as identified by ORDER.orderNo. The value of an order (ORDER.totalValue) is the sum of the cost of each item in the order (as identified by ITEM.price).
CUSTOMER.totalOrders and ORDER.totalValue are not directly entered by a user of the application - they are derived fields.
For an RDBMS, these table definitions must be entered into the data definition part of the database (or schema) before SQL can be used to add, modify and retrieve records.
MUMPS does not enforce the use of a data-definition scheme and, as such, records can be written directly to the MUMPS data store without the need to formally defined the data. However, it is important to note that a relational schema can easily be layered on top of an MUMPS data repository for the purpose of accessing the MUMPS data via SQL-based reporting tools. A relational schema can be retrospectively added to an existing MUMPS data store provided the records are modeled to (approximately) third normal form.
The tables can be represented in MUMPS using the following globals:
CUSTOMER
^CUSTOMER(custNo)=name_”~”_address_”~”_totalOrders
ORDER
^ORDER(orderNo)=custNo_”~”_orderDate_”~”_invoiceDate_”~”_totalValue
ITEM
^ITEM(orderNo,itemNo)=price
The relationship between the CUSTOMER and ORDER tables could be represented by another global:
^ORDERX1(custNo,orderNo)=””
This would provide pointers to all Order Numbers for a specified Customer Number.
In MUMPS, its up to you what global names you use. Also you have a choice of either using different globals for each table (as we’ve shown above), or using the same global for some or all the tables and indices. For example, we could pack everything together using the following global structure:
^OrderDatabase(“customer”,custNo)= name_”~”_address_”~”_totalOrders
^OrderDatabase(“order”,orderNo)= custNo_”~”_orderDate_”~”_invoiceDate_”~”_totalValue
^OrderDatabase(“item”,orderNo,itemNo)=price
^OrderDatabase(“index1”,custNo,orderNo)=””
For the purposes of the examples that follow, we’ve chosen to use separate globals for each table.
Adding new records to the Database
Let’s start with a really simple example. Let’s add a new customer record to the customer table.
SQL
INSERT INTO CUSTOMER (CustNo, Name, Address)
VALUES (100, ‘Chris Munt’, ‘
MUMPS
Set ^CUSTOMER(100)= “Chris Munt”_"~"_"
We are using the tilde character (~) as the field delimiter. We can, of course, use any character we like for this purpose (including non-printable characters). For example, we could use:
Set ^CUSTOMER(100)= “Chris Munt”_$c(1)_"
Read the $c(1) (aka $char) function as meaning “the character whose ASCII value is 1”. This second alternative therefore uses ASCII 1 as a field delimiter.
Of course, in a real situation, the data values would be passed to the insert query (or MUMPS command) by means of a program as values of variables:
SQL
INSERT INTO CUSTOMER (custNo, :name, :address)
VALUES (:custNo, :name, :address)
MUMPS
Set ^CUSTOMER(custNo)=name_"~"_address
Retrieving records from the database
SQL
SELECT A.name, A.address
FROM CUSTOMER A
WHERE A.custNo = :custNo
MUMPS
Set record=$get(^CUSTOMER(custNo))
Set name=$piece(record,"~",1)
Set address=$piece(record,"~",1)
Note the use of the $get(). This is a convenient way of safely retrieving a global node. If the node doesn’t actually exist, the $get() function returns a null (“”). If you don’t use the $get() function, ie:
Set record=^CUSTOMER(custNo)
Then if the specified global node doesn’t exist, MUMPS will return a run-time
Set record=$g(^CUSTOMER(custNo))
Removing records from the database
SQL
DELETE FROM CUSTOMER A
WHERE A.custNo = :custNo
MUMPS
kill ^CUSTOMER(custNo)
Note that this simple example doesn’t take referential integrity into account. You’ll see later how we’ll deal with this.
Parsing the database
SQL
SELECT A.custNo, A.name, A.address
FROM CUSTOMER A
MUMPS
s custNo=”” f s custNo=$order(^CUSTOMER(custNo)) Quit:custNo= “” do
. Set record=$get(^CUSTOMER(custNo))
. Set name=$piece(record,"~",1)
. Set address=$piece(record,"~",2)
. Set totalOrders=$piece(record,"~",3)
. ; add code here to process the “selected row”
Note the use of the so-called “dot syntax”. The lines preceded with a dot (“.”) represent a subroutine, called by the “do” command that terminated the first line. Typically you would do whatever you need to do with each “selected row” within the subroutine, as indicated by the comment (the last line that starts with the semi-colon (“;”).
Using MUMPS functions to encapsulate database access
In practice, and in order to maximise code reuse, the MUMPS queries shown previously would usually be implemented as functions. Examples of such functions are shown below.
Adding new records to the database:
setCustomer(custNo,data) ;
If custNo="" Quit 0
Set ^CUSTOMER(custNo)=data("name")_"~"_data("address")
Quit 1
This function could be now called repeatedly in the following way:
kill data ; clear down data local array
set data(“name”)=”Rob Tweed”
set data(“address”)=”
set custNo=101
set ok=$$setCustomer(custNo,.data)
Note the period (“.”) in front of the data parameter. This is a “call by reference”. data is a local array, not a simple scalar variable, so we must pass the name of the array by reference into the function.
The example above assumes that the setCustomer function is contained in the same MUMPS routine as our example run-time code. setCustomer might be held in a different routine (eg ^myFunctions), in which case our run-time code would look like the following:
kill data ; clear down data local array
set data(“name”)=”Rob Tweed”
set data(“address”)=”
set custNo=101
set ok=$$setCustomer^myFunctions(custNo,.data)
The $$setCustomer() function is known as an extrinsic function. It returns 0 (ie false) if a null value of customer number is specified. Otherwise the record is saved and it returns 1 (ie true). You may want to test the value of the variable ok to check that the function worked.