D3 Syschk Description and Examples

Description

checks a system running under Unix to detect 'abnormal' situations.

The "syschk" utility starts a phantom process which periodically checks the system is behaving 'normally'. If a system parameter goes beyond a threshold, defined by the System Administrator, a message is sent to one or more users, and, optionally, an entry is put in the "errors" file. All elements are optional. The following elements are controlled:

Unix Swap: When "syschk" is started, it always ensures that the Unix swap (or paging) space is at least equal to twice the physical memory. Then it checks periodically the swap usage does not go beyond a predefined level (90% of the total available space by default).

Total System CPU: The percentage of the CPU spent in System mode (Kernel, drivers, etc...) must stay below a predetermined level (25% by default).

Runaway Processes: Each time a sample is taken, the CPU time of the active processes is controlled. If a process has consumed more than a predefined percentage of the sampling period, a warning is issued. For example, if the sampling period is 10 minutes (600 seconds), and if the process consumed more than 5% of this time (30 seconds, which is an enormous amount of CPU), this process is probably in an abnormal tight CPU loop. "syschk" displays the Unix status ("ps") and the result of a "where", if it is a Pick process. If a process exceeds the limit more than three times in a row, the reporting of the error stops. This is to ensure that the System Administrator will not receive constant messages for a process which is running a large report, for example. If the process is still running after nine samples, then the reporting restarts three more times, and the reporting cycle restarts.

Unix File System Usage: The state of a predetermined list of Unix file systems is controlled, to make sure they do not get full (used over 90%). This is to prevent Unix crashes due to the filling up of critical file systems. By default, only '/' is controlled.

Overflow Usage: When the used Pick overflow exceeds a predetermined percentage of the total overflow space, a message is generated. The default level is 90%. Overflow is reported only once a day, at the first sample taken after noon.

Basic Usage: This monitors the usage of the FlashBASIC "basic" area. When the used "basic" space exceeds a predetermined percentage of the total "basic" space, a message is generated. The default level is 90%. To examine the "basic" space in more detail, use the "shpstat" command.


This command must be run on the "dm" account. Only one instance of a phantom process running "syschk" is supported at any given time.

The form "syschk start" starts the syschk command running as a phantom with the same argument as the last time it was started. If necessary, the process is stopped before it is restarted. If the process was never started, a set of defaults is applied.

The form "syschk edit" enters the Update processor to allow editing the arguments used by the "start" command.

The form "syschk now" instructs the "syschk" phantom process to take a sample immediately, and send the anomaly messages, if any, to the the terminal requesting the sample, instead of the normal notify mechanism.

The form "syschk stop" stops the syschk command running as a phantom.

The form "syschk" without any argument, simply reports whether syschk is currently running and displays when it was started, the parameters and some of the current system parameter values.

With one or more arguments, "syschk" creates a phantom process and returns immediately to TCL. Arguments can be specified in any order.

"keyword{=value}" specifies which system parameters to control:

sampling=[ sec | hh:mm:ss ]
Specifies the sampling delay. The delay can be expressed either in seconds of in hh:mm:ss format. If "sampling" is not specified, the default is 30 minutes.

notify=[ user{,..},!n{,..}, /dev/ ttyXX{,...} ,* | OFF ]
Specifies the list of Pick users, of Pick port number or Unix devices to notify in case of abnormal situation. The users can be specified as a list of explicit Pick user ids, Pick port numbers, in decimal, prefixed by an exclamation mark (!), Unix devices, prefixed by a slash (/), or any combination. If '*' is used, all users logged on the system are notified. If a period ('.') is used, it is replaced by the tty name. If 'OFF' is specified, notification is disabled. If "notify" is not specified, or if the specified users are not logged on at the time the anomaly occurs, a message is sent to "dm" or "sysprog". If a Unix device is specified, it must exist and be writable when "syschk" is started.

syscpu{= percentage {%}}
Specifies the maximum percentage of total CPU usage Unix is allowed to spend in System mode. If 'percentage' is not specified, the default is 25%.

proccpu{= percentage {%}}
Specifies the maximum percentage of CPU a process is allowed to take. If 'percentage' is not specified, the default is 25%. This trigger point may be a little difficult to evaluate. For example, on a system with only one active user running a Pick/BASIC cpu intensive program, the process will, naturally, takes 100% of the CPU, since there is no other running process. To avoid false alarms, select a sampling period large enough. It is probably unusual to have a process doing 100% of CPU for 15 minutes.

swapusg{= percentage {%}}
Specify the acceptable swap usage at any given time. 'percentage' is the amount of swap actually used. If 'percentage' is not specified, a swap usage above 90% of the total swap will be considered abnormal.

ovfusg{= percentage {%}}
Specify the acceptable overflow usage at any given time. 'percentage' is the amount of overflow actually used. If 'percentage' is not specified, an overflow usage above 90% of the total Pick space will be considered abnormal. Errors are reported only once a day.

basicusg{= percentage {%}}
Specify the acceptable 'basic' usage at any given time. 'percentage' is the amount of 'basic' space actually used. If 'percentage' is not specified, a 'basic' usage above 90% of the total 'basic' space will be considered abnormal.

diskusg= filesystem{,filesystem,..}
Specify the list of Unix file system which should never get full (over 90%). The list should always include '/'. Depending on the system, '/usr', '/tmp' might have to be included. Some Unix system, will not be able to boot with a full '/'. If not specified, '/' is the only Unix file system to be checked.

log
Log messages in the "errors" file. This keyword is equivalent to the (L) option.

nolog
Do not log messages in the "errors" file. This keyword is equivalent to not having the (L) option. It supersedes the (L) option.

When started as a phantom, "syschk" runs for ever, until the system is shutdown. To stop it, use "syschk stop".

Syntax

syschk keyword{=value} {...} {(options} syschk edit syschk now syschk start syschk stop syschk

Options

F Start syschk in foreground on the current process, as opposed to a phantom process. With this option, the only way to stop the process is to do a break / end, or a logoff.

L Log a short summary of the error messages to the "errors" file. The initial swap space control is always logged. Can be specified by using the "log" keyword.

Q Quiet. Suppresses some user messages ("started", "stopped")

V Verbose. This option can be used if "syschk" is run in foreground, instead of a phantom.

Example

syschk sampling=00:30:00 syscpu=10% 

         notify=/dev/tty0,bob,!0 

         swapusg=60% proccpu=10% log

  Start a phantom process to check the system every 30 minutes for a CPU system 

usage above 10%, a swap usage above 60% and a process runaway limit of 10%. In 

case of anomaly, a message is sent to the Unix terminal '/dev/tty0', 

the Pick user 'bob' if he is logged on, and to the line 0 whether it 

is logged on or not, and a short message is logged in the 'errors' 

file. The other parameters are left to their default values.

syschk

  Check whether syschk is running. This would display, for example:

    syschk is running on port 132

    Started on 03/11/94 at 08:20:21

    Current running parameters:

      Sampling period           00:30:00

      Notify list               /dev/tty0  bob  !0

      Maximum system CPU %      10

      Maximum CPU % per process 25

      Maximum % of swap         60

      Maximum % of overflow     90

      Unix file systems         /

      Log messages (0=no;1=yes) 1

    Current System Status:

      User CPU usage            3%

      System CPU usage         11%

      Waiting for IO           82%

      Idle CPU                  4%

      Total swap space         128 Mb

      Used swap space           76 Mb (59%)

syschk stop (q

  Stop the phantom running "syschk" as a background process, 

suppressing the message "stopped". This command could be included in 

the "user-shutdown" macro.

syschk start

  Restart the "syschk" phantom with the same parameters.

syschk edit

  Edit the syschk command line to change the arguments. Use the Update 

processor command to edit the command line.

Terminal "Blink Mode" issue explanation and resolution

There are several things going on here, so I'll try to explain them one at a time. 

First, Visual Styles is simply a way to associate a border style and foreground / background color combination with a visual attribute (dim, reverse, etc.) Basically, you can assign Dim to have an Inset border style, Reverse to have a Flat style, Underline gets a Raised style, etc. 

The color assignment is basically the same as it has been with earlier versions of Accuterm, except that there are now some color schemes that change some of the standard Accuterm palette colors to system colors (as assigned in the Display properties, Appearance tab). For example, black is remapped to "Window Text", white is remapped to "Window Background", etc. 

Clicking the "Classic Windows" or "Windows XP" under "Visual Styles" simply sets up the palette using appropriate system colors and sets a color combination and border style for each attribute, ensures that "Border Style" is not set to "None" and disables true underlining. You can customize any of the attributes to suit your own application style. 

The way Visual Styles is used is to simply display data using an appropriate attribute. For example, using "Classic Windows", text displayed in the Reverse attribute will appear as if it were a Windows text box, using the same colors and border style used by other Windows programs. 

Now, to answer the question of why changing the attribute changes everything on the screen: this is how a real Viewpoint terminal works, at least regarding attributes. The old Viewpoint A2 terminal uses a single bit in each character as a "tag". Each tagged character is displayed in the currently selected visual attribute. When the current attribute is changed, say from Reverse to Dim, all tagged characters on the screen are redisplayed in the new attribute, Dim in this example. We call this "tagged" attributes in the Accuterm documentation. 

When using the Viewpoint A2 Enhanced emulation (or the Wyse 50 emulation), you can use both "tagged" attributes and "embedded" attributes. In these emulations, the tag bit is also used to indicate a "protected" character. 

"Embedded" attributes take a character position on the screen. The "embedded attribute" character itself is displayed as a blank without any attribute. All the characters the follow the "embedded attribute" character are displayed using that attribute, at least until another "embedded attribute" character is encountered. This continues to the end of the screen. This is why Wyse terminals, unless carefully programmed, will flicker. 

Typical terminal coding would suggest that you do something like: 

NORMAL = @(-14)

REVERSE = @(-13)

PRINT @(10,7):REVERSE:'This text is in reverse video':NORMAL

When you run this code, you may see the screen flicker. This is because when the terminal receives the REVERSE code, it changes the attribute of all characters from the cursor position to the end of the screen to reverse video. Then the text is displayed (in reverse video). Finally, when the terminal receives the NORMAL code, it changes the attribute of all characters from the cursor position to the end of the screen to normal video. During the time after receipt of the REVERSE code but before the NORMAL code, the screen is in reverse video from position @(10,7) to the end of screen. When the NORMAL code is received the portion of the screen from @(40,7) to end of screen is changed back to normal video, causing the screen to flicker. 

Flickering is not as much of a problem with fast network connections as it used to be with serial ports, but it still can occur and looks very unprofessional. 

The correct way to program for this kind of terminal is to print the attribute codes in reverse order: print the NORMAL code first, then the REVERSE code: 

NORMAL = @(-14)

REVERSE = @(-13)

PRINT @(40,7):NORMAL:@(10,7):REVERSE:'This text is in reverse video'

This resolves the flickering problem because the attribute changes only until another attribute is encountered. In this example, NORMAL is at the end of the field, so when the REVERSE code is received by the terminal, only the area from the cursor position @(10,7) to the next attribute @(40,7) is changed to reverse video. 

To further confuse the situation, when Wyse released the Wyse 60 terminal, they abandoned the "embedded attributes", and switched to "character attributes", which required two whole bytes for each character, instead of only a single byte. "Character attributes" are similar to "tagged attributes" in that as each character is displayed, it takes the current attribute in effect. It is different in that changing the current attribute does not affect anything already displayed. "Character attributes" do not occupy a screen position like "embedded attributes" so you have more freedom in screen layout. And there is no flickering when using "character attributes". 

Now for some fun: when ADDS released the 4000 series of terminals, they attempted to include the same features as a Wyse 50 running in Viewpoint/A2 mode with "Enhance" turned on (hence "Viewpoint A2 Enhanced"). However, they must have recognized that "embedded attributes" were not a good solution, and while supported (sort of), they added "character attributes" as well. A different Escape sequence was used for the "character attributes", so the 4000 actually supports all three attribute schemes. 

The current version of Accuterm 2K2 fully supports all three attribute schemes in the Viewpoint A2 Enhanced emulation. If possible, use the ADDS 4000 "character attributes". The escape sequence for character attributes is ESC:'g':ATTR where ATTR is the same as the Wyse attributes shown in Appendix A of the Programmers Guide. This is basically the same sequence as used by Wyse 60, except for the lower-case "g" rather than upper-case "G". 

Hope this explanation was not too confusing!

How to use D3 "EREPLACE" to help with variable character swaps in strings

I came across an issue where I needed to convert a single (1) character to two (2) characters and the normal CONVERT statement just did not cut it.

So I found this little command buried in the D3 docs called “EREPLACE”

It will convert a variable length string to another variable length string in a record, array or string.

Use:

In your code, you have to put in “$OPTIONS EXT” (without the quotes obviously) preferably in the top of the program where you might have $INCLUDE for instance.

Then you can use this command like this:

NSTRING = EREPLACE(“The new time is…”,”new”,”oldest”)

PRINT NSTRING

Which will print “The oldest time is…”

I hope you can use this in your D3 adventures.  

There might be other tricks out there, but this worked pretty well and it was very quick.

D3 AutoStart in Linux (CentOS & RedHat)

We no longer use /etc/inittab to auto boot D3.   It was replaced by a new subsystem called Upstart. 

There is now a directory called /etc/init.   We must place our d3 boot configuration file in there.

vi /etc/init/d3.conf

# D3 Database start on last console tty12 (alt F12)

description "D3 database server"

start on stopped rc RUNLEVEL=[2345]

exec /usr/bin/d3 -0 -a x -t tty12

Once you save the d3.conf file, you can start D3 one of 2 ways:

1) Reboot and it will start, a good final test

2) Use the command "start d3" and it will execute this new d3.conf file.

To see if it is really running issue this command: 

# initcl status d3

d3 start/running, process 5547

To stop d3, just login to D3 and issue the normal shutdown command at TCL. Also, there is d3 -k in Linux but only use this as a last resort.

# start d3

d3 start/running, process 5547

# initctl status d3

d3 start/running, process 5547

MultiValue Database Versions

D3

Rocket D3 Database Management System is a powerful multidimensional database management systems that are the choice of thousands of application developers worldwide.

Powerful development environment designed for today's business critical transactional and analytical database applications, providing:

• Scalability
• Flexibility
• Seamless interoperability
• Connectivity
• Low system management cost

For more information, visit http:/​/​www3.rocketsoftware.com/​rocketd3/​database/​index.jsp

OpenQM

OpenQM is a high performance multivalue database environment with a low price tag and close compatibility with other products. Available for Windows, Linux, Mac, AIX and PDA, a commercial QM licence includes the AccuTerm terminal emulator at no extra cost. The QMClient API allows development of VB, C and web front ends.

OpenQM is the only multivalue database available both as a fully supported commercial product and in open source form under the General Public License.

For more information, visit http:/​/​www.openqm.com

UniData

Power and Performance without the Maintenance.

Deliver Even Better Availability, Interoperability and Security for Your Data.

UniData is an extended relational data server ideal for embedding in a variety of industry-focused solutions. Its extended relational model, with NoSQL access to data, is perfect for rapid cost-effective applications serving the needs of businesses from SMB to the enterprise where outstanding performance, reliability, scalability and manageability are needed.

Product Highlights:

• Delivers low TCO and exceptional scalability to meet the challenges of on demand business in the SMB and large enterprise markets
• Provides intuitive design and high-performance queries using an extended relational data model
• Supports security and robustness via encryption, authentication and high availability features
• Features native high-performance interfaces for .NET, Java and more
• Includes U2 Dynamic Objects for easier exchange of data with web services

Emerging Technologies:

• Data Type Enforcement provides an optional check on data written to a file in order to assure only valid data is stored.
• Replication with External Database Access (EDA) can be used to replicate your data to a SQL database on the same server.
• U2 MetaData Manager (U2 MDM), a user-friendly tool, quickly and easily builds and maintains mappingsfor your SQL tools accessing UniData.

For more information, visit http:/​/​www.rocketsoftware.com/​products/​rocket-​unidata

UniVerse

Fast and Flexible for the Enterprise.

Modern data. Managed.

UniVerse is an extended relational data server optimized for embedding in vertical applications. UniVerse offers a nested relational (MultiValue) data model , with NoSQL access to data, means intuitive data modeling and fewer resulting tables.

Product Highlights:

• Delivers high performance to business solutions through super-efficient database structures
• Speeds your time to market while providing a lower total cost of ownership
• Advanced management with audit logging and account-based licensing
• Protects your data and provides around-the-clock availability to your enterprise application
• Integrates easily with third party software and external data sources

Emerging Technology:

U2 MetaData Manager (U2MDM), a user-friendly tool, quickly and easily builds and maintains mappings for your SQL tools accessing UniVerse.

For more information, visit http:/​/​www.rocketsoftware.com/​products/​rocket-​universe

jBase

A multidimensional database, a development environment including a development language, and a middleware component allowing other mainstream and standards-based products to communicate with the jBASE products. The unique jEDI middleware enables access to other databases such as Oracle, DB2 and SQL Server. Microsoft Windows, all major Unix platforms including Linux, and IBM's e-Servers are supported.

For more information, visit http:/​/​www.jbase.com/​products/​jbase.html

MultiValue Database - Description & History

MultiValue Database

MultiValue is a type of NoSQL and multidimensional database, typically considered synonymous with PICK, a database originally developed as the Pick operating system.

MultiValue databases include commercial products from Rocket Software, TigerLogic, jBASE, Revelation, Ladybridge, InterSystems, Northgate Information Solutions and other companies. These databases differ from a relational database in that they have features that support and encourage the use of attributes which can take a list of values, rather than all attributes being single-valued. They are often categorized with MUMPS within the category of post-relational databases, although the data model actually pre-dates the relational model. Unlike SQL-DBMS tools, most MultiValue databases can be accessed both with or without SQL.

History[edit]

Don Nelson designed the MultiValue data model in the early to mid-1960s.^[1] Dick Pick, a developer at TRW, worked on the first implementation of this model for the US Army in 1965. Pick considered the software to be in the public domain because it was written for the military. This was but the first dispute regarding MultiValue databases that was addressed by the courts.[1]

Ken Simms wrote DataBASIC, sometimes known as S-BASIC, in the mid-70's. It was based on Dartmouth BASIC, but had enhanced features for data management. Simms played a lot of Star Trek while developing the language, in order to have the language function to his satisfaction.^[2]

Three of the implementations of MultiValue: PICK version R77, Microdata Reality 3.x, and Prime Information 1.0, were very similar. In spite of attempts to standardize, particularly by International Spectrum and the Spectrum Manufacturers Association, who designed a logo for all to use, ^[3] there are no standards across MultiValue implementations. Subsequently, these flavors diverged, although with some cross-over. These streams of MultiValue database development could be classified as one stemming from PICK R83, one from Microdata Reality, and one from Prime Information.^[4] Because of the differences, some implementations have provisions for supporting several flavors of the languages. An attempt to document the similarities and differences can be found at the PRDB ^[5]

Marketing groups and others in the industry over the years have classified MultiValue databases as pre-relational, post-relational, relational, and embedded, with detractors classifying it as legacy. It could now be classified asNoSQL. With a data model that aligns well with XML and that permits access with or without the use of SQL.

One reasonable hypothesis for this data model lasting more than 40 years, with new database implementations of the model even in the 21st century is that it provides database solutions in a big bang for the buck fashion. Historically, with industry benchmarks tied to SQL transactions, this has been a difficult hypothesis to test, although there are considerable anecdotes of failed attempts to get the functionality of a MultiValue application into arelational database framework.

In spite of a history of more than 40 years of implementations, starting with TRW, many in the MultiValue industry have remained current so that various MultiValue implementations now employ object-oriented versions of Data BASIC, support AJAX frameworks, and because no one needs to use SQL (but some can) they fit under the NoSQL umbrella. In fact, MultiValue developers were the first to acquire nosql domain names, likely prior to other database products classifying their offerings as NoSQL as well. MultiValue is a seasoned data model, but with so many vendors competing in this space, it has been constantly enhanced over the years.

Data model example[edit]

In a MultiValue database system:

• a Database is called an "Account"
• a Table is called a "File"
• a Column is an "Attribute" composed of "Multivalue attribute" and "Subvalue attribute" to store multiple values in the same attribute.

Data is stored using two separate files. A "File" to store raw data and a "Dictionary" to store the format for displaying the raw data.

For example, assume there's a file (table) called "PERSON". And in this file there is a attribute called "eMailAddress". The eMailAddress field can store a variable number of email address values in the single record.

So the list [joe@example.com, jdb@example.net, joe_bacde@example.org] can be stored and accessed via a single query / disk read when accessing the associated record.

To achieve the same (1-to-many) relationship within a traditional Relational Database System one would be required to create an additional table to store the variable number of email Addresses associated to a single "PERSON" record. However, modern Relational Database Systems do support this data model too. For example, in PostgreSQL one can set up a column of type Array of baseType (baseType being any PostgreSQL data type).

MultiValue DataBASIC

Like the Java programming language, the typical DataBASIC compiler compiles to P-code and runs in a P-machine. It has as many different implementations (compilers) as there are MultiValue databases.

Like PHP programming language, the DataBASIC language does all the typecasting for the programmer.

MultiValue Query Language

Known as ENGLISH, ACCESS, AQL, UniQuery, Retrieve, CMQL, and by many other names over the years, corresponding to the different MultiValue implementations, the MultiValue query language differs from SQL in several respects. Each query is issued against a single dictionary within the schema, which could be understood as a virtual file or a portal to the database through which to view the data.

LIST PEOPLE LAST_NAME FIRST_NAME EMAIL_ADDRESSES WITH LAST_NAME LIKE "Van..."

The above statement would list all e-mail addresses for each person whose last name starts with "Van". A single entry would be output for each person, with multiple lines showing the multiple e-mail addresses (without repeating other data about the person).