|
| · |
polycosmic-zervertest.zapto.org |
| Home | · | History | · | Downloads | · | Full Help Text | · | Contact Us | · | About | · | Admin |
NetZerver Documentation - Help Text | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
On each of the admin web pages that is used to configure and manage a NetZerver, there is a link for "Help". Along with the data field names and description that should be self-evident, this text also tries to describe what each data field means, recommended usage, interaction with other features, and information for problem analysis. The help text for each admin web page, and some other notes that needed to be put somewhere, are included here.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
What is a ZipZerver ?
What is a ZipZerver ?
What a ZipZerver is not --
Steps for Initial Set-up for a ZipZerverUpdated 2019-Sept-17
Admin Default PasswordWhen a ZipZerver is set or reset to factory-default conditions, the admin password is "admin" (without the quote marks). Under some conditions, a ZipZerver can be booted up and running, but the ZerverModule, or other flash memory, is not accessible. This will result in a state similar to factory-default, however the admin password will be "noflash" (without the quote marks). This is done to signal that something very basic has gone wrong with the flash-device method. Under other conditions, it might be possible that neither the password-file with the admin:admin password, nor the file with the admin:noflash password can be established. If so, the admin password may be set to "bigproblem" (also without the quote marks).
How to Install the Firmware2019-Nov-11 The easiest way to install this firmware to a new system is to be handed a USB flash-drive which has been formatted as a Config-Device with firmware installed. Adjust the BIOS or EFI parameters for boot-device order to boot-up from the device. An alternative method for firmware install uses a Bootable-CD. There may be several versions of such a CD,
Initial boot-up During this first boot-up, if done from a Bootable-CD, there might be a pause of 15 seconds or so while looking for a Config-Device. There may also be a pause, perhaps as long as two minutes, while the httpd server awaits enough entropy for secure operation, and Linux observes various timing events that provide a certain amount of environmental noise. During this first boot-up, an attempt will be made to acquire DHCP addresses for the network interfaces. The values found for IPv4 and/or IPv6 will be displayed on the system console. (If there is no DHCP server and a default static IPv4 address of 10.10.10.10 is selected, see below for a system-console method that can apply arbitrary static IPv4 addresses.) With an IP-address, use http to connect to the system. With a Config-Device set to factory-default values, login to the admin web-pages with admin:admin (userid of "admin" with a password of "admin"). You can proceed to the directions in these Help Notes marked as Steps for Initial Set-up. If the boot-up was done from a Bootable-CD, login to the admin web-pages with admin:noflash (userid of "admin" with a password of "noflash"). Everything is functional and you could proceed with Steps for Initial Set-up. However, without a Config-Device, those configuration choices are only in RAM and will be lost on a reboot. A Config-Device will need to be created. A USB flash device of 512MB to 16GB will be suitable (and even larger devices can be used). The admin web page System -> SW Update -> Create Config-Device can be used to find, format, and initialize a suitable Config-Device, and the Help-link on that page has much more to say. The result will be a reboot in which the Bootable-CD must be ejected and the system BIOS or EFI parameters adjusted to boot from the new device. No DHCP Server, Static IPv4 needed for First Bootup If the initial boot-up cannot get suitable DHCP addresses, the system might settle on 10.10.10.10 as a fallback address. There is a console method to assign static addresses.
The rest of the configuration should be done using http connection to the system.
Access Permissions and Limits2024-Jan-19 (for nfs restriction note)
Release 1.2.19 Notes2024-Dec-03
Release 1.2.18 Notes2024-Oct-26
Release 1.2.17a Notes2024-Jul-26
Release 1.2.17 Notes 2024-Jul-23
Release 1.2.16 Notes2024-Apr-17
Release 1.2.15 Notes2024-Jan-25
Release 1.2.14 Notes2023-Nov-08
Release 1.2.13 Notes2023-Oct-11
Release 1.2.12 Notes2023-Sept-27
Release 1.2.11 Notes2023-Aug-31
Release 1.2.10 Notes2022-May-31
Release 1.2.09 Notes2022-Jan-29
Release 1.2.08 Notes2022-Jan-21
Release 1.2.07 Notes2021-Nov-02
Release 1.2.06 Notes2021-July-02
Release 1.2.05 Notes2020-Feb-07
Release 1.2.04 Notes2019-Dec-27
Release 1.2.03 Notes2019-Sep-17
Release 1.2.02 Notes2019-Aug-28
Release 1.2.01 Notes2019-July-25
Release 1.2.00 Notes2019-May-28
Release 1.1.17 and Earlier Notes
General Setup - Admin Passphrase and Network NameUpdated 2024-Nov-17 for logfile size limit changes This web-page is one of the first to be encountered when doing set-up for a new ZipZerver. For a summary of all the important steps, see section Steps for Initial Set-up for a ZipZerver As set at the factory, the initial password for user "admin" is "admin" (without the "quote-marks"). Although this is a secret, it is possible, perhaps, given enough time, that someone could guess this value. Please change the admin password to a more challenging value at your earliest opportunity. Several password values are actively discouraged, such as "123", "nimda", or returning the value to "admin", and will be met with a snippy rejection if selected as the new password. There are times, however, when a poor password is better than no password (or the default password), and so the checkbox Apply New Password even if weak can be set, and the lame password will be allowed through (and which can still fail -- in some cases setting the new password to match exactly the old password will be rejected in the lower-level routines). Note that the "weak password" filter is dazzlingly unsophisticated. Getting through it does not mean that your new admin password is a good password. You can use letters, numbers, spaces, special characters. The maximum length is somewhere around 120 characters. Insert standard advice here about names of children and pets, words in a dictionary, and spouse's birthday. After a successful admin password change, you will be prompted to login again with the next admin web page that you select. Network Name for this Zerver is one of the key values during setup, and it can be set on the System -> General page as well as the Network -> TCP/IP web page. Some proposed names will be rejected, in the interests of getting a network name that can be used by all of the supported network protocols. The field Location of this Zerver is used as one of the values within SNMProtocol. It is also handy to include with e-mail alert messages to help identify something about the physical location of the Zerver. The value System Message of the Day, if set, will appear after login for FTP users, for SmartMirror users, on HTTP share-list and data-access web pages, and on each admin web page as one of the possible nag-lines (after "admin password has not been changed" and before "No flash device found"), and could be used to help identify which ZipZerver is being accessed, announce operation plans (e.g. "Reboot Friday night at midnight for hardware upgrade"), or legal mumbo-jumbo (e.g. "Unauthorized access is prohibited. Also, coffee is hot"). The two submit-buttons Save Next/Config Value and Save and Apply Now appear on several of the web pages for network protocol configuration. There might be a number of changes which should all take effect together that must be set on separate admin web pages. If so, the values can be selected or entered, and Save Next/Config Value used to save the values with the configuration data, but do not attempt to "Apply" them. The button Save and Apply Now is used to save the new values with the configuration data (as above), and immediately "Apply" those changes, and any other pending configuration changes that had been Saved but not Applied. In general, the ZipZerver does not need to be re-booted after a name change, or other network changes. It will not hurt the ZipZerver, however, if you go and reboot all of your Windows systems. Note that during a change of Network Name -- for this and most of the other admin web pages, the flow of control is to produce the top of the page with the title, navigation links, and the banner using the old Network Name, and then process the change request.
To duplicate Syslog Major Events, Login History, and/or Current Details to another ZipZerver (or other system configured to receive Remote Syslog events), two fields must be set: Remote Syslog system address (as an IPv4 address or DNS-name) and Send to Remote Syslog system to select which set of messages to send. This may allow multiple ZipZerver to accumulate operation information at a single system (for ease of monitoring). (Using an IPv6 address does not appear to work.) For this ZipZerver to be the receiver of those messages, the field From Other Systems must be set to Listen, Save Messages from other systems, along with setting those other systems with the IPv4 address or DNS-name of the current ZipZerver. (What was possible with earlier versions of this firmware -- two ZipZerver could send their messages to each other, so that after hardware catastrophe for either, diagnostic information may be available on the other. This used to work because systems that both send and receive events would not forward events received remotely. That arrangement now fails, with the initial message passed back and forth furiously. Unclear (and mostly not investigated) as to why. This would be an easy way to test what happens if a runaway program making major event syslog messages manages to run the /flash file-system totally out of space.) For using Remote Syslog messages to diagnose problems, you can draw conclusions from the messages that you see. Because the mechanism is UDP-datagram, rather than TCP-stream, there is a higher chance of lost messages, depending on network traffic and the number of routers between systems. Thus conclusions of the form "The problem is X, because otherwise there would be Syslog Message Y" are less strong when examining Remote Syslog messages, rather than local Syslog messages. Messages sent by Remote-Syslog will remain on that system until over-written or discarded on that remote system. That is, local discard of local syslog messages will not cause them to be removed from the remote. The Size Limits of several log files can be changed within the section for Login, Current, WebAccess log size limits. Depending on how busy a system is, there may be advantages to logging somewhat more (or less) http and https traffic in the WebAccesLog, other protocol login and logout activity in the LoginLog, or general system activities in the CurrentDetails log. This display shows the different log files, whether in /flash (to maintain continuity over reboot) or in the /var file-system in RAM (which are discarded and restarted with each reboot). If a value set in the Size Limit field for a log is too large, warning messages are given for various boundaries. For a system running in the factory default configuration showing 6148 (or so) blocks (1024 bytes) free for the /var file-system, setting Size Limit field to 6148 for either CurrentDetails or WebAccessLog would (eventually) cause the /var file-system to run out of space, which will have a serious negative impact on system operations. Since it could be a long time before that occurs, the value is allowed and you are encouraged to adjust to be somewhat smaller. The soft_limit number is based on the /var file-system space needed for running a reasonably heavy-duty test load with 400 local userid and 30 Active Directory Domain userid, each with permission to a share and to several of the userid Other Operations. The intention is that the log files can describe the state of the system as it has been recently. That is in contrast with security log files, which need to address questions such as "Have we ever seen traffic from this IP-Internet-Address?" and "Have we ever seen requests of this particular form?" Logs for a ZipZerver can be useful, but are not designed for that purpose. The Major Events log is expected to accumulate entries at a much slower rate, and uses a different mechanism to ensure that the file does not grow without bound, and thus cannot have the Size Limit changed the same way. As items of importance during initial setup, this web page also contains links to the pages to Agreement and Licenses and Register and Support.
Set Timezone, Date, Time, Auto-Time ParametersUpdated 2015-June-24The ZipZerver expects to operate with UTC -- Universal Co-ordinated Time, formerly known as GMT -- Greenwich Mean Time. This reflects long-standing Unix (and now Linux) tradition, that as an inherently multi-user networked remote access system, the timezone desired by an administrator looking at system logs and the timezone desired by a user were not necessarily the same. The result were systems which ran using UTC and each user, including an administrator, could set a timezone to be used for his purposes. Changing the timezone for display purposes for one user does not change the underlying time-keeping mechanisms. The primary use for the system date and time is to tag entries made in the system log. When used as a file server, the ZipZerver stores files and their attributes, including date and time, as set by the client system. The fields to Set Date and Time for Local Time Zone will set the date and time for the running system, and will also set the "hardware clock" which continues to run while the system is off, for use with a later system-start. Even through the system will be running using UTC, the values are entered using time in the currently selected timezone. To Set Local Time Zone a number of values are immediately available for regions of the US and for UTC (listed as GMT0). For other regions there is a lengthy pull-down list with time-zones listed alphabetically, organized by continent, ocean, and country -- Africa, America (both North and South), Antarctica, Asia, Atlantic, Australia, Europe, Indian, Pacific, and US. A more entertaining alternative to setting the system date and time is to discover, set, and maintain the current date and time from one or more time servers, using the fields with Automatic Time Set/Synchronization. The choice for Continuous monitor and adjust will start a time-monitor program, and will also start that program at the next system re-start. The auto-time program will attempt to contact the configured time-servers, judge which one provides the best time service, and then synchronize the system date and time to it. Under controlled conditions, the time can be kept synchronized within 128 milliseconds. The Set time once choice will go through many of these same steps, but after establishing an idea of what the current date and time is, the program ends. When run from the admin web page, this process can take part of a minute (e.g. 10 to 25 seconds), and then will display the result log. This is valuable for testing. If the value Set time once has been set at system startup, the configured time servers will be contacted, and the system date and time set. When the time has been set this way, the system date and time can drift. So where are these time servers to be found? Within an organization which manages machines using DHCP, the message which assigns an IP address to a machine such as the ZipZerver can also provide an address of a time server. If so provided, that time server should be used and can provide a time-standard for all local machines. There are spaces to configure up to four time servers, either by IP address or by DNS name. One possibility, as shown at https://www.ntppool.org/en/use.html, the DNServer for "pool.ntp.org" returns IP addresses from a large pool of servers that have volunteered to provide time service. Each time that you request an address for "0.pool.ntp.org" or "1.pool.ntp.org", etc, you may get a different address, thus directing the load to a wide range of time servers. These are mostly stratum 2 and 3 servers, with the occasional stratum 1. Because these are widely distributed, this would not be the carefully controlled conditions that achieve 128 millisecond accuracy. Another possibility is time1.google.com through time4.google.com as described at developers.google.com/time/guides. Google provides "Leap Smear" when the unreliability of the earth as a time-keeper requires adding a "Leap-Second" (or taking one away). Standard time servers are supposed to have seconds from 23:59:00 to 23:59:60 in the 61 seconds up to the leap-second, clearly a candidate for least-tested time-reporting feature. Google time, in contrast, adjusts the length of a second from noon to noon around a Leap-Second adjustment, "Smearing" the time and avoiding a minute with 61 seconds (or only 59). A system that gets its time from a mix of standard and "smeared" time servers will have internal disagreement of up to 0.5 seconds for a 24-hour period around a leap-second. A future major motion picture involving a bank heist that depends on the time-uncertainty is almost certain. To display something about the quality of the time service, in particular, that at least one of the configured remote time servers is still alive and well and appears to be operating properly, the stratum value will be shown with the Current Status for Automatic Time Set/Synchronization. If the currently chosen time reference server is operating at stratum=N, then the value for the ZipZerver will be N + 1, and values from 2 through 4 (or so) will be fine. If all of the configured remote time servers are unreachable, then the stratum will be 16, and the local time on the ZipZerver will not be corrected by reference to any external server. Additional information can be shown as Status of Time Monitor and Upstream Time Servers. Along with the stratum value, one table ("peers") has a row for each remote time server showing the IPv4 or IPv6 address with one-character to indicate "*" for the selected time reference, "+" for a reachable server that is a candidate to be promoted as the selected reference if needed, and "-" for a remote server that is reachable which has observed behaviour such that it is not a good candidate to be promoted. The "refid" column for each row shows where that remote server gets its time service, intended to avoid having multiple servers that appear to be independent, but which could have a common failure-point or weak-point. For a stratum=1 remote server, that may be listed as ".GPS." or another direct hardware time source. For stratum=2 and beyond, the refid will show another IPv4 or IPv6 address. The "st" column shows the stratum for each remote server. The other values for each row show type as "u" (for unicast, rather than broadcast), when the last poll was received (in seconds), the poll-interval (seconds), an 8-bit value showing which of the previous 8 polls received a response (in octal, going through 1, 3, 7, 17, 37, 77, 177, and 377 if there is always a response), and the values that keep track of the remote-server's usability. An additional table ("associations") shows much of the same information, in which the rows of the two tables correspond. The "index", "assid", and "status" values are internal values. The column "conf" shows if the peer was configured (versus ephemeral assocations). Column "auth" shows if authentication is in use with that remote server (which ZipZerver code does not currently use). The column "reach" shows if the server is reachable or not (and may be more readable than the octal bit pattern 377). The "last_event" column shows the most recent step in the attempts to reach and to judge the usability of each server. The "condition" column shows the judgement among the various servers of which to use. One value should be sys.peer for the currently selected remote time reference (shown with "*" character on the peers display). Other values are candidate, outlyer, and reject. If there are no usable remote time servers, the stratum value will be 16, the "condition" column will not have a row with sys.peer, the "reach" column will show "no", "last_event" of unreachable, or other indication of why each configured remote time server is unusable. If one or more of the remote time servers can be brought back on-line (or reachability restored), the time-monitor will notice (eventually -- the poll interval can be 1024 seconds), and synchronization will be restored. Failing that, Automatic Time Set/Synchronization can be switched off, other remote time servers configured by DNS name or IP address, and then Continuous monitor will restart the time-monitor. If the remote time servers were found using pool.ntp.org DNS names, then the restart will (almost certainly) get a different set of time servers. As an exercise in arbitrary customization, the display format for date and time values can be chosen, as shown by the list under section Set Date-Time Display Format. This will affect the date-time display within various of the ZipZerver admin web pages (however, the date and time-stamp within the Status section log files is not affected). As a further exercise in even-more arbitrary customization, you can compose your own date-time display format by selecting the empty data line, and creating your format using the %-elements as illustrated by the other choices, or as documented within the Linux library function "strftime()" (and also shown by the documentation for a Linux "date" command).
Alert Configuration and SelectionUpdated 2022-Sep-13E-mail alerts can be sent for various system conditions, and can be sent on a regular schedule to indicate that all is well -- system is up and operating and is able to send status e-mail. Recent experience (2022) has been that systems with full-featured e-mail handling, with expert configuration and operation, have encountered e-mail delivery problems. It appears that the mysterious collective that decides how (or if) e-mail should work are dealing with spam by increasing the conditions by which an e-mail can be deleted without any kind of error indication. This rather adds to the difficulty of configuring and debugging an e-mail setup, even for those with a heavy-duty set of e-mail tools. The ZipZerver attempts to be light-weight, has limited e-mail capabilities, and very likely can be configured to provide useful messages on a regular schedule, right up to the point where they stop for no apparent reason. Reliable delivery, especially in the face of a spam problem that keeps evolving, adapting to every counter-measure, cannot be assured. Note that an e-mail alert will be sent, or attempt to be sent, when a condition is found, but if it cannot be sent within a few seconds, the e-mail is abandoned. Unlike most e-mail handling systems, messages are not queued for transmit at a later time. The Alert Configuration web page, when used to send a Test-Message, will display the log of the protocol elements with the SMTP Server. This is intended to aid in debug of the Alert setup. The SMTP Mail Server will identify the first e-mail server in what could-be a long chain of servers involved with delivering a message. This may be part of the immediate organization or company, may be part of the ISP-Internet-Service-Provider infrastructure, may be a server somewhere on the internet, and in some cases may be a server which is close to the intended recipient. The form can be --
A given SMTP server might support all of these possibilities, or only one, or a different :port-number altogether. The DNS name of the SMTP server, such as smtp.cox.net, can be used provided that one or more DNServers are defined on the TCP/IP configuration web page, and such a server must be accessible and provide prompt mapping to the IP address at the time the e-mail is to be sent. The event system-shutdown, for example, will not be delayed by more than a few seconds by an attempt to send an e-mail alert, DNS lookup included. The Encrypt checkbox appears next, since this is the order it would be handled within the protocol -- after connection to the SMTP Server and before any sensitive information is sent, in particular, before any Mail Server Userid and Password is sent. If checked, a STARTTLS request will be sent to the server. If accepted and successful, the rest of the session will be encrypted, providing authentication of the name of the SMTP Server as well message security for the first hop in transmitting the message. The Encrypt operation can be rejected or fail --
It is unclear what can be done to verify that the session is, in fact, encrypted and the name of the SMTP Server has been verified. The details of a message when delivered include "Received:" header-lines which are supposed to be pre-pended by each server which handles the message. Sometimes there will be mention of the encryption-type used to receive the message (e.g. "Received: ... using TLSv1.2 with cipher ECDHE..." ), and sometimes there is not. In any case, the encryption you select would occur between the ZipZerver and the SMTP Server, which is the first step of what may be several or many steps before delivery. This will protect the userid and password that may be needed with this first server. Whether encryption is used by any of those other steps is not your choice. (The more robust method for e-mail security is end-to-end encryption, perhaps using PGP-format messages, perhaps using program gpg, which allows a message to be encrypted so that it can only be decrypted by the intended recipient, regardless of the path or the number of steps for it to be delivered. This system does not currently use end-to-end encryption for e-mail alerts.) The fields Mail Server Userid (if needed) and Mail Server Userid Password are now available to identify the system to the SMTP Server. These fields might not be needed if the SMTP Server is the destination needed for all of the receiver addresses, or if the Server has been configured (some would say mis-configured) to allow forwarding to any other address. If the connection is encrypted, the password will be protected. The password field cannot handle Tab- or Enter-characters (which are likely interpretted by your browser), but can handle space and displayable characters, including "-double-quote, '-apostrophe, `-grave-accent, $-dollar, and \-backward-slant, but because it is web-based input, a value with the combination of "-double-quote followed by >-greater-than can show partly scrambled results upon redisplay of the web-page. Generally systems that handle e-mail will keep trying to deliver a message, perhaps over several hours or days, using queue and retry mechanisms. This system is not one of them. If the message cannot be forwarded to the SMTP Server within a few seconds, it is abandoned. The value Operation Timeout (sec) can be used to select how many seconds. If the SMTP Server is on the immediate LAN and is unlikely to be overloaded, 10 seconds may be plenty. This time is most directly visible during System Shutdown. If the SMTP Server is, in fact, not available, the shutdown operation will appear to hang while attempting to send the System-Shutdown alert. The default of 30 seconds can seem awfully long.
The Sender E-mail Address field may have to be set to appease the SMTP server -- it had to be valid_cox_userid@cox.net to get through the cox.net server. This field might also be set to help recognize or filter the e-mail alert message -- sending from "dasher@legal.domain.name" might identify the ZipZerver named "dasher" from some number of other nodes. Keep in mind that only a very few organizations need just one ZipZerver. Within the fields Receiver E-mail Address and Additional Receivers as many as three e-mail addresses can be given. These should be addresses in minimal form -- user_name@dns.name Values that will work in these fields depend on the network characteristics and limitations, and interact with each other in various ways.
If you must figure out a suitable SMTP Server for the Receiver Address, several tools might be available.
It may be that Additional Receivers cannot be used, unless they are in the same domain as the first Receiver. (The handling of multiple receivers in different domains, alternate SMTP servers, queue and retry, are all features automatically handled by a more full-featured e-mail facility.) Even with the SMTP log displayed when sending a Test Message, there can be debug challenges. When testing with a valid@acm.org address, the log showed text to the effect 450 Recipient reject, Greylisting challenge offered, with mention of "normal" SMTP behavior which would wait several minutes before a retry. Extensive experiments were not done in this state, but perhaps a retry within a few seconds would have resulted in the attempted-sender address being added to a blacklist. The ZipZerver does not do automatic retry for e-mail messages, so another web-page operation for a Test Message was done 5-6 minutes later, which must be within bounds for "normal" SMTP behavior since the message was accepted and additional messages since then were delivered for many months. To aid in recognizing or filtering by subject line, the Add Node Name to Subject Line check box can be set. For the conditions in which to send an e-mail alert, a few notes. Disk and Raid-Group Event is intended to be the critical category, including Raid-Group problems found, recovery-begun, and recovery-complete. The messages in category Disk and Raid-Group Information are expected to be non-critical conditions for which an event-log entry occurs, and so an e-mail alert can occur. Examples of the non-critical messages -- during build or re-synchronization of a Raid-Group, when progress has passed 20%, 40%, 60%, and 80%. Informative, entertaining, perhaps useful, but non-critical. A particular Disk and Raid-Group Event should get some elaboration. The Alert message text DeviceDisappeared will occur for a raid-0-striped or jbod Raid-Group very shortly after its creation or after being put on-line. Program mdadm monitors Raid-Groups looking for a condition, such as single-drive failure in a Raid-Group with redundancy, that could go unnoticed because the Raid-Group remains available and fully usable. The DeviceDisappeared message means that the raid-0-striped or jbod Raid-Group has been dropped from the set of Raid-Groups to monitor. Without redundancy, a single-drive failure will not go unnoticed since the Raid-Group will be dead. File System Low Space can be set to trigger at various levels, and works by periodically running a task which checks the amount of free space. This means that a ZipZerver could go from sufficient space, perhaps 11% free, to no space, and the alert might not be sent for nearly an hour. The File System Space Monitor condition will generate a report on file-system space available on a regular schedule. A sequence of such e-mail alert messages can be used to monitor that --
Software/Firmware UpdateUpdated 2019-June-09 for digital signatureUpdated 2019-Nov-11 for /flash2 From time to time, there may be new/better software/firmware available for your ZipZerver. The Software/Firmware Update web page is used to install such an upgrade. Two problems can occur with any product with Software/Firmware Update capability.
Your ZipZerver tries to solve these possible problems by keeping two or more complete versions of the Software/Firmware available. The "current" Software/Firmware is always the default choice when your ZipZerver is re-started, however, the "previous" version, and sometimes a version marked as "old", are available after an update, and can be selected from the system console at a suitable moment during the re-start sequence. If there have already been two or more downloads when new Software/Firmware is to be downloaded, then one of the old versions will be deleted. Generally, the Software/Firmware that had been known as "previous" will be deleted, and the "current" Software/Firmware will be renamed before the new Software/Firmware is downloaded. Upon operational catastrophe (e.g. power-off in the middle of the update operation), there will still be workable Software/Firmware available for selection at system-start-time, either under the name "previous" or "current", depending on the nature and timing of the failure. Upon software engineering catastrophe (that is, the new/better Software/Firmware does not start), and recovery using the Software/Firmware marked as "previous" (selected at the right moment from the PC console during startup), the failed Software/Firmware is known to have never been successfully started (it is known internally as "next" rather than "current") so that when a Software/Firmware set must be chosen for delete, the failed set can be chosen. The number and identity of available bootload file-sets can be found using the Bootable File-Sets link on this web page. There are several ways that the Software/Firmware update operation can fail (in addition to random-moment power-off) --
Starting with Version 1.2.01, there is code to check for a digital signature as part of the update firmware. The firmware image contains several PGP-format public keys, including one from "Zerver Development". A digital signature uses an even-more heavy-duty checksum over the essential components of the firmware. The signature is created from that SHA checksum and the private-part of the key-pair. The firmware uses the public-part of the key, included with the firmware, to verify that all components are present, uncorrupt, and the signature had to have been created by the private part. This would provide a very high-level of assurance that the firmware update was provided by the same organization/group/person/mob that created the currently-running firmware (the one with the public keys included). In a world in which internet-available firmware-update files have been known to be compromised in various ways, frequently to create a security back-door, this assurance can be a very good thing. Thus the choices available on the Software Update web-page. Selecting Proceed with firmware update only if the file has a digital signature from firmware development provides the highest level of assurance that the next firmware to be loaded has been created by someone with access to the private-part of the public-key included in the current firmware. Continuity is assured. The new firmware might not be the work of geniuses or saints, but it was, absolutely, created by people who are not much worse than those that created the currently running firmware. (Which, of course, you are attempting to replace....) The choice Require digital signature from development or other public key built-in ... recognizes that organizations can change. There are a handful of public keys included with the ZipZerver firmware. Zerver Development is one. Others are expected to be involved with creating Register-Support Certificates. A firmware file signed by such an auxillary key has a higher level of assurance of authenticity than firmware with no digital signature at all. And the choice Allow firmware update from any source, do not require digital signature is present for several reasons.
Beginning with version 1.2.04 there is a capability to create a new Config-Device. The SW Update operation is part of that. If a new Config-Device has been formatted and is mounted at /flash2, any SW Update operation will be directed to that new device, rather than to the current Config-Device mounted at /flash. This is intended to allow a wider range of firmware versions to be installed. If this occurs, "/flash2" will appear as part of the success message. If that was not intended, go to System -> SW Update -> Create Config-Device and select operation Unmount a newly-created Config-Device and then repeat the SW Update operation.
Bootload File Set Flash CheckThe ZipZerver can have more than one set of files for loading at system start-up, to be able to recover from software update operations that have failed, or for which the newly loaded software is a catastrophic step backwards. Briefly during system startup, the system console will show a menu of available bootload choices -- one or more among "Zerver_current", "Zerver_previous", and "Zerver_old" (and under some conditions, "Zerver_next"). There can also be these choices with "_tty" appended, to select a bootload file that will use the first serial port for system output, rather than the console. The Bootload File Set Flash Check web page shows what bootable file-sets are available with the ZerverModule™ flash device, and information about each file-set, including the version identifier and the length in (1024)K blocks. There is a heavy-duty checksum, from program "md5sum", created for each file-set as part of the download file, checked to ensure that transport from software development through download and install has been done properly. This web page checks that value, re-computing the current actual checksum and comparing it to the expected value. From time to time there has been a suspicion that a file-set used for bootload has become corrupt, with mysterious Raid-Group or system failures that have gone away after a firmware update with the same firmware version. This web page allows for a check of such a condition. If such corruption is found, this web page allows the failed file-set to be deleted. Also, there are ZipZerver with limited-size configuration flash, sometimes a software update can fail if space is too tight, and delete of an old version might be required. On first entry to this web page, there can be a delay of several seconds before the bootable file-set information appears, since reading the full files in each set takes time, to compute the current actual checksum. On redisplay of this web page, the result may occur much more quickly, if the files are still held in buffers by Linux. (The files of the file-set used during bootload are loaded into memory during the bootload process, but that is done by the bootload firmware which is starting Linux.) The operation to delete a selected bootable file-set will allow you to delete all the file-sets. In general, this is not recommended, but may be necessary for some odd-sized configuration flash device. If the last bootable file-set has been deleted, then SW Update can be done to install another one. If a reboot is initiated before SW Update can be done, the reboot might be successful -- the bootload map will still be pointing to the blocks of the deleted file-set. If those have not been over-written by other files, the reboot might work. Not recommended. The message "Problem -- Cannot access directory with bootload files -- /flash/boot" can occur if the ZipZerver is bootloaded from a CD (or other device) and there is no ZerverModule, or if the ZerverModule is not recognized during the system startup sequence.
Format Device for Configuration and FirmwareUpdated on 2021-June-01 for BIOS/EFI choiceThe facility to Format a New Configuration Device allows for several possiblities --
There are several choices on this web page that will eventually lead to the Select Operation below with Format the above-selected Config-Device ... and the Begin Selected Config-Device Operation Now submit button. Some of the other Select Operations will be described in the text below. Config device should be created with EMPTY configuration should be selected if the new Config-Device should appear with "factory default" values, such as admin:admin (initial login with userid admin with password "admin"), no other users, no share definitions, no SmartMirror Tasks, no web-Certificates, no ssh-keys. The choice ... should be created as BACKUP will create a Config-Device with all of those values from the currently running configuration. Use this if the actual situation is initial install of the firmware after boot-up from a Bootable-CD. During the reboot, the system BIOS or EFI boot-order parameters will need to be adjusted for the device. The choice created as BACKUP can also be used to create an actual backup Config-Device, providing a level of safety similar to the System -> Config Save operation, which creates an encrypted file with all of the configuration details, as saved on a Raid-Group. An additional caution after creating a BACKUP Config-Device -- do not leave it installed on the machine with the original Config-Device during a reboot, since the boot-up process sometimes discovers disks in a different order. The start-up sequence will check for something that looks like a Config-Device, and will be content when one is found, which might not be the one intended. Major Events and Login History syslog files will not be present. Any configuration changes, such as new users or permission changes, will seem to vanish if the next reboot happens to select the original Config-Device. So do not leave a newly created BACKUP Config-Device installed in a system until it is needed. (Note that there is a Neutralize the device currently mounted as /flash operation below, but it cannot be used to resolve this since it will leave the (intended) backup as not-recognizable as a Config-Device at the reboot when it is needed.) (Deliberate testing of the backup Config-Device is a good idea. If two USB flash-keys are involved, during reboot, remove the original Config-Device and leave the backup device in place. Verify that the configuration choices are in place, such as users and share permissions, then reboot again, putting the original back in place. You may need to deal with BIOS or EFI boot-device choices at each step.) The list to Pick New Configuration Device will let you select a disk-like device that is unused, that has no partitions defined. A newly-inserted USB flash device might come with some pre-defined Windows-like partitions, which can be deleted at web-page Storage -> Devices. If the devices on the system are left-over from other testing on which Raid-Groups had been created, enough pieces might have been found to be assembled into a Raid-Group. If there does not seem to be an available device, you might have to go to Storage -> List RAID Groups web page to take an accidentally assembled Raid-Group offline and then stopped, and then use the Devices web page to delete any partitions. Firmware Set to Install may offer several choices. If the system was booted-up and is running with a Config-Device, there may be several firmware versions that are available, marked as next, curr, prev, and/or older, each with a detailed version identifier. If the system was booted-up from a Bootable-CD for installation, and does not currently have a Config-Device, none of those will be listed. That CD does not need to remain in the CD-reader during operation, but if it is still present, one of the Select Operation choices below, Mount an available Boot-Install-CD can be done, which may then show zervsw_64.tar and/or zervsw_32.tar as available choices for a Firmware Set. It could be that none of these choices for Firmware Set are available or appropriate, and No firmware choice should be chosen. There are other possibilities. After a new Config-Device has been selected, formatted, and initialized, it can remain mounted (and will be listed as "/flash2" within the list of devices). If it is still mounted when a System -> SW Update operation is done, the firmware will go to /flash2, rather than /flash, the current-in-use Config-Device. This method can be used if the Firmware to Install is different from what is listed, such as _32/_64-bit difference with the intended system or that the desired Firmware is newer or has some custom changes that are not available in the Firmware Set list. Also, it is legal to create a Config-Device with no Firmware Set at all, if the intention is to boot-up always from a Bootable-CD. The original design of this system used a small flash device, perhaps not high performance, for the firmware and configuration choices, with all parts of the actual disk devices available for the user's data. Small flash devices have been getting bigger. The Size of Config Partition on Device choice can select a size smaller than the full device. The space required is dominated by the size of the firmware which has grown from 30MB to 50MB over several years, and might grow to 80MB over enough time. To recover from SW update catastrophe, there can be up to 4 versions, selectable during a short window during boot-up. Thus 512MB should be a safe size choice. The 128MB choice could be used by a system that always boots from a CD and needs no copies of the firmware. The simplest choice is Maximum size allowed from device (Up to 16GB), which will use the entire device, if 16GB or less (a boundary which is used to try to simplify the scan of devices during boot-up to find candidates for Config-Device). The choice Boot-up Mechanism, older-BIOS or EFI is clear if the intended hardware has EFI-Enhanced-Firmware-Interface startup firmware and there is no choice for 'Legacy' or 'BIOS-Compatability' mode -- then EFI must be chosen (and is the reason why this choice had to be added). After that, the best choice is murky. The EFI choice has extra BIOS-like mechanisms that appear to be recognized and workable in at least one BIOS system -- VirtualBox running in 64-bit non-EFI mode. So there is some reason to hope that a device with an EFI choice can be used in either environment. And if it turns out that the EFI mechanisms do not work for some older BIOS only hardware, the BIOS choice will create systems exactly as they have been in the past. The intention is that either boot-up choice for the new device, when used in either mode, after a SW Update and reboot, there will be a short time-out to select Next/Current firmware to run, or to select Previous. In BIOS mode, lilo (or isolinux for bootable-CD) will present the choices. In EFI mode, program refind.efi (usually documented as rEFInd) will present choices for both flash device or bootable-CD. EFI bootup can be very complicated, and can fail (or be absurdly clumsy) in amazing ways. See this section for longer bootup bios and efi rant. If there is Extra Space in Device beyond Config Partition, it can be left unused or a simple (one-drive) Raid-Group can be named and created. This file-system can be used in almost all of the ways that any Raid-Group can -- define share points, take off-line for a file-system check, stop the Raid-Group, and reassemble to put back online. However, if the Raid-Group is stopped and the Storage -> Devices web page is used to delete the partition, there is currently no way to reclaim and reuse that space. For the Select Operation and the Begin ... Now submit button, the primary operation is to Format the above-selected Config-Device ... which will do a few completeness and consistency checks and then create the Config-Partition on the Config-Device, which can take many seconds as the firmware is installed. If a Raid-Group is to be created in any extra space, that operation is started and may continue for a minute or more in the background (similar to how the Create RAID Group web page works). In the Device List, the Config-Partition of the new Config-Device will be listed with "/flash2". The operation Mount an available Boot-Install-CD, mentioned above, can be used to list zervsw_64.tar and/or zervsw_32.tar firmware choices that may be available for install from a Bootable-CD. The Unmount a Boot-Install-CD operation should be used before ejecting the CD, since otherwise the buffer system can become confused. (An explicit Unmount is not needed if a reboot is to be done, such as when a Bootable-CD is being used for an install to create the Config-Device. The CD must still be ejected and the system BIOS or EFI boot-device order must be adjusted as part of the reboot.) The Unmount a newly-created Config-Device operation must be used before the device is physcially removed, to ensure that the buffer system has pushed all writes out to the Device. Unmount is a good idea even if the device does not need to be physcially removed, since any SW Update operation done when /flash2 is mounted will go to the newly-created Config-Device, not to the current Config-Device. (Note that before the device is physically removed, if there is a Raid-Group in the Extra Space, it must also be put in a safe state. Use the Storage -> List RAID Groups web page to Take File-System Offline and to Stop Raid-Group.) After the Unmount, the device will still be in the list of devices, but will no longer be listed as "/flash2". If the new device was created as part of an initial install and there is to be a timely reboot, then the Unmount step is not needed. The Neutralize the device currently mounted as /flash operation is expected to be useful to return a system to no-Config-Device found, factory-default state which is needed if there is a test script for the handling of an initial install and test scripts which should start in factory-default state, and maybe occurs less frequently in other contexts. It can be used in a situation with competing Config-Devices in which the wrong one has been selected, but at the expense of eliminating its possible use as a Config-Device after reboot. Thus, if a BACKUP Config-Device was created, left in place over a reboot, and found first, this option can be used to declare that upon reboot, it should not be found as the Config-Device. Which means that it will not work as a BACKUP Config-Device. The Undo ... Neutralize is present mostly because it was cheap to add and seemed like it might be useful, especially to those who felt Neutralize should be a solution to the multiple conflicting Config-Device problem, and then carefully read through these notes. Last, and certainly least, No Config-Device operation needed will mostly have the effect of redisplay of the page, and any recently inserted USB devices might appear.
Configuration, Task, and Userid System Data Save and RestoreAlthough set-up and operation of a ZipZerver is intended to be simple and straight-forward, it is not likely to be so quick and simple that re-doing it would be a joy. The admin web page Configuration, Task, and Userid System Data Save and Restore is intended to allow a single file to be created which holds a copy of all of the files needed from flash memory to be able to restore normal system operation. This file is written to a local Raid-Group (or to a share-point within a local Raid-Group) so that its content is protected by RAID or by SmartMirror operation. This is intended to protect against several possible disasters --
In each of these cases, the Raid-Group, File-System, and data may be okay, but restore of access to the data cannot occur until there is full re-configuration of the replacement flash device or of the system with the backed-up data. If done step-by-step, this re-configuration could take hours and be error-prone. The data which is maintained in flash which would need to be restored --
All these need to be restored for there to be full-access by all users to all data with all protocols. In addition, there are --
All of this configuration, task, and userid system data can be restored in one easy step. Provided that a Config-Save file has been created at an appropriate moment. Creating a Config-Save File You can select any Raid-Group or any share as the destination for the back-up configuration data. Use the RAID-Group or Share for Config-Save File(s) selection for where the Config-Save File should go. Selecting a share (rather than a Raid-Group) has the advantage that the Backup file can be visible on the network, such as with SmartMirror, so that the Backup file or files can be replicated. Selecting a Raid-Group (rather than a share) has the advantage that during recovery from a catastrophe, the Backup files in a Raid-Group are immediately visible to a replacement ZipZerver board, but a share (which was not at the base of a Raid-Group) would have to be re-defined before the Backup file would be visible for use with a restore operation. Use the Set Directory for Config-Save Files submit-button to select the directory, which will produce a list of the files in that directory with names that might already be Backup files. To create the Config-Save File, create a passphrase which will be used to encrypt it, and hit the Set Directory and Create New Config-Save File Now button. You should see the result file, with the current date and time, in the list of possible recovery files. The file that will be created will be named ZipZerverBackup.NODENAME, using the current actual node-name. If there had been a file with that name, it will be kept, renamed to be ZipZerverBackup.NODENAME.old The result file is normal, in terms of ownership and permission, and can be copied, moved, or deleted. If a Config-Save File is renamed, it is a good idea to keep the text "Backup", with capital-letter "B", as part of the name, so that the file will be listed as a candidate to select for restore. Why a Passphrase for the File? The two Passphrase entry input-fields indicate that a new passphrase is being established which will be used to encrypt one new Config-Save File. This passphrase can be the same or different from the current admin passphrase, and can be the same or different from the passphrase used for any other Config-Save File. Different has advantages, provided the value needed can be recalled weeks or months later (or found in the "Critical Passphrases" folder held in the safe maintained in the Security Office by the guy who always worries about disaster recovery). What is being protected by this passphrase, and why --
Although the ZipZerver is not really intended to be used in a hostile security environment, it does have some capability to allow access by some users and restrict it to others. This capability can be seriously weakened by improper protection of a Config-Save File. Since there are times when a weak passphrase is better than no passphrase, there is a check-box to Apply New Passphrase even if weak, which will bypass some checks that look for particularly weak passwords. Those tests are not exhaustive -- if you create a passphrase that is accepted without having to use the "weak" checkbox, it is not necessarily a "strong" passphrase. For internal technical reasons, a text string had to be chosen to act as an internal delimiter, with the result that the string could not be used as a passphrase. Thus, "weak" checkbox or not, you cannot use "password" as a password. Recovery Steps In a recovery situation, use the RAID-Group or Share for Config-Save File(s) selection to select the location where the recovery file is to be found. In a serious recovery situation in which all the share-definitions have been lost, it may be necessary to create a temporary share-definition to the directory with the Config-Save Files. After submit with the Set Directory for Config-Save Files button, any files that have names that might be Config-Save Files will be listed below. The files listed will be those with "Backup", the name "ZipZerver", or the current node-name as part of the filename. The file-date and size are also shown to help identify each file. Select the desired file from the list. The Passphrase for Selected File is the value used when the Backup file was created. It might or might not be the same as the current admin passphrase, and might or might not be the same as the admin passphrase that will be restored. After considering again whether this is, indeed, the Config-Save File that you want, hit the Restore from File and Reboot Now button. If the passphrase is valid for the file, and the file contents appear to be a valid Config-Save File, then flash will be restored and a reboot will be started. Note that after system restart, the set of network interfaces will be the same. The set of disk-devices will be the same, as will the name of any Raid-Groups and the content of the File-System. However, there may be many changes in the state of the system. The admin passphrase needed will be the one in effect when the Config-Save File was written. Depending on the nature of the recovery being done, the IP-addresses may be different (if STATIC addresses had been used or come into use), the node-name may be different, the network protocols enabled may be different, the set of userid may be different, the passphrase that would be needed for each userid would be the value in effect when the Config-Save File was written, the share definitions may be different, the share-userid-permissions may be different, the set of SmartMirror tasks may be different, the certificate for https may be different, and the key-identity for ssh access will be the one in use when the Config-Save File was written.
Shutdown or RestartThe Shutdown or Restart admin web page has a number of practical uses.
The IP-Internet-Protocol address of the web-client which initiates a shutdown/restart is noted, along with other identifiers such as the reported browser type, to be used in any e-mail alert about the shutdown, and which can also appear in any e-mail alert generated at the subsequent restart. For the choice "Restart the system", under some conditions there will be a display of the list of Shares after the restart. Whether this occurs depends on timing of the restart, and the behavior of the web-browser (in persistence and retry for a timed refresh directive). By popular demand, there is a check so that a refresh of the web page with the message "System Restarted" at an unfortunate moment does not produce an extra, un-intended, system restart. Even though the choice says "Shut Down the system", note that an actual power-down might not occur, depending on the power-supply type, BIOS/EFI configuration, and kernel module configuration. Currently, the general result is that the ZipZerver does a clean shutdown and then does succeed in dropping power. Depending on the hardware (and perhaps on some BIOS configuration choices), the power might not shut off. If so, you have to recognize when it is safe to force the power to be off --
Network Configuration and Server ProgramsThe admin web page Network Configuration and Server Programs shows the Current Status and configuration choices for all of the network server programs. This page and the TCP/IP admin web page are intended to give a comprehensive picture of the network configuration for a ZipZerver. Configuration changes for all of the protocols shown can be made in one step with the controls on this page. There are some additional values that may be found on the web pages for individual protocols that are not shown here, generally for clutter-appearance reasons. For each protocol, Enable has choices of either Yes or No, and has Current Status as either off, started, or stopped. There are security advantages to disabling any and all protocols that are not needed. For some of these protocols, there may be performance or resource-use advantages for disabling a protocol (which will be mentioned in the Help for that protocol). On the other hand, it is convenient to have all protocols enabled that might have any use. Note that if a protocol is off or stopped and is later needed, it can be re-enabled with little difficulty, and without the need for a system restart. Enable or Disable of each protocol is independent of the choice whether the protocol would be allowed for a given Share. There can be a Share or Shares for which this protocol would be allowed, even though the server for the protocol has been disabled and is off. Also, this protocol might be enabled and using some system resources even though there are no Shares which are allowed to be accessed using it. The submit-button Save Config/Next Values will save any choices from this page to the configuration file, but does not attempt to Apply them. Any values listed in the column Current Status should be unchanged. The submit-button Save and Apply Now will save choices from this web page to the configuration file, as above, and attempt to Apply them and any other pending configuration changes. The values listed in the column Current Status will show the result. The submit-button Save, Apply, and Join Windows Domain Now will save the choices to the configuration file, as above, and attempt to Apply them, as above, and also attempts to Join a Windows Domain, as described in the Help for the admin web page for Windows. Changes made and Saved, but never Applied, will stay in the configuration file indefinitely, and would be applied with a system restart, if there is ever a restart.
TCP/IP Internet Protocol ConfigurationUpdated 2023-Sept-10 for IPv6 Gway notesThe admin web page for TCP/IP Internet Protocol Configuration shows the low-level Ethernet network interfaces, the associated IPv4 and IPv6 Internet-Protocol addresses, and related values. The ZipZerver Name can be set here, as well as with a field on the web page System -> General, where it might be more convenient, since setting the Network Name of a Node is likely to be nearly as important as replacing the default password. From one to eight Ethernet interfaces can be handled, marked as eth0 to eth7, and discussed below as ethN. For each Ethernet port that has been found, several values are displayed and several values can be set. The MAC-Media-Access-Code is the 48-bit unique value for the interface, expressed in 6-bytes of hexadecimal. The current status for each interface can include the text no carrier if there is no ethernet cable, the cable connects to a switch without power, or the cable is not right between cross-over and straight-through. If there is carrier on the port, the current status may show a speed in Mbits/sec, such as 100, and may show the phrase full duplex if connected to a switch or half duplex if connected to an ethernet hub. IPv4 addresses are 32-bits, expressed here in "dotted-decimal" notation, with 4 integers, each from 0 to 255, (such as "192.168.1.1") so that each int can express all the values for 8-bits. The 32-bits can handle around 4-billion addresses, which may not be enough for a planet with around 8-billion people. IPv6 addresses are 128 bits, generally expressed in 16-bit clumps of hexadecimal digits, separated by ":"-colon characters (such as "2001:db8:0:0:1234:5678:9abc:def0", in which the left-most run of two or more consecutive zero clumps can be replaced with "::" -- "2001:db8::1234:5678:9abc:def0", which does not save that much ink in this particular case). The 128-bits might not be enough to handle the entire galaxy, or last long enough to get the earth up to the red-giant phase of the sun, but it should last for quite a while. (Note that to cover 8-billion addresses, one needs only to go from 32-bits to 33-bits.) IPv4 and IPv6 can be configured independently of each other, with the exception that OFF for either protocol will shut down the interface for both. Use choice NONE if one of the protocols should be (largely) unused. The ethN_type and ethN_6type fields are the major choices for each interface. The IPv4 choices will be described first.
ethN_6type will select the (mostly) independent choice for IPv6 addresses. Some differences --
Multiple static addresses and syntax changes -- The Slackware (development) system used as a base of ZipZerver firmware had a network script that allowed multiple static addresses, for both IPv4 and IPv6. That turned out to be exactly what was needed when a test system that had one assigned IPv4 static address needed to be changed to another. There was an interval when responding to either address was what was needed. To specify more than one static IPv4 or IPv6 address, enter the values in the ethN_ipaddr or ethN_6ipaddr fields, separated by ","-comma character. Unclear if those distinct addresses could be in disjoint network-addresses with different sizes, but if so, the syntax recognized within ethN_ipaddr and ethN_6ipaddr will support it -- ethN_ipaddr: 192.168.33.170/24, 192.168.33.171/24 ethN_6ipaddr: :2001:db8:0:123::1111/64, 2001:db8:0:456::2222/64 The field Other Gway provides one more opportunity to specify an IPv4 Gateway-router, and field IPv6 Gway to specify one for IPv6. One or both can be left empty if DHCP or STATIC ethN_gway fields, or IPv6 ROUTERADVERTISE message, has provided a suitable value for use by the node. A problem seen in which IPv6 addresses did not seem to be working was fixed with IPv6 Gway, defining a router with the network address (the high-order bits, typically 64) from the IPv6 static address, with all-zero for the low-order bits, specified with "::" characters at the end. If there are multiple ethN interface with different IPv6 network addresses, a local net wizard can provide guidance as to which network has the appropriate IPv6 gateway. At this level of the internet, routing for a packet can be over-simplified as --
Routing is done by the machine as a whole, not by each individual Ethernet interface. For full network access, the machine needs at least one Gateway-router for each protocol in use, IPv4 and IPv6. Each individual Ethernet interface could have one defined or not. Any IP-addresses given for a router must be directly accessible -- it must be within the LAN addresses of at least one of the Ethernet ethN that are expected to be available. DNS Name Servers, if specified at all, must be given by IP-address, and allow DNS-Domain-Name-System names, like "pool.ntp.org" used by the Date-Time service, to be resolved to an IP-address. One or more values for this field may have been provided by a DHCP Server or the DHCP6 selection, if used for any of the Ethernet ethN ports. This field is not required, and may be left empty. Unlike the IP-address for a Gateway-router, which must be directly reachable, the IP-address for a DNS Name Server must be reachable in some number of steps, and does not need to be reachable in one step (provided that a suitable Gateway-router is defined). Unlike most of the other fields on this web page, the DNS Name Server values can be either IPv4 or IPv6. As with the configuration of high-level network protocols shown on the admin web page Network -> Information, there are columns marked Current Status to show the current values being used for each of these fields, and a column marked Config/Next Value into which new values are put or selected. Also like the other Network control admin web pages, there is a submit-button marked Save Config/Next Values that will Save the specified values with the configuration data, but does not attempt to Apply those values. The Current Status values should be unchanged. The Save and Apply Now submit-button will Save the values, as above, and attempt to Apply them immediately. The Current Status values will show the result of the Apply-attempt. Note about changing an IP-address while using that IP-address -- Success in this case is announced by a partial web page display, with the browser display hanging, progress bar perhaps marching toward a time-out, waiting for more output and connection-close from an IP-address that no longer exists. This is normal for Apply Now for IP-address change which starts using the IP-address to be changed. A "Stop Loading" control for the browser, if any, can be used to abandon the wait for the connection, and then you can try connecting to the expected new address. If the new address is not known, as might occur for a change from STATIC to DHCP, you can try browsing to a newly-assigned ZipZerver Name. Note that attempted contact to a new, unknown IP-address for a previously existing Network Name may be frustrated by the number of helpful agents that have cached the previous-Name, previous-IP-address relationship, with time-outs from a few minutes to many hours. As of Release 1.1.13, there is a ZerverView-like web page, called ZView, which may be used to discover any ZipZerver from another ZipZerver on the same LAN segment. The ZView web page may be able to display some information about all of the ethN ports in use. There can be a more conventional success-display if the IP-address to be changed is on another interface, and is not involved in the connection for the admin web page itself. If an ethN_type of STATIC has been set using wrong values, or if a machine has been moved from one network location to another and good STATIC values no longer work, the recovery can be tricky. If the value of the incorrect IP-address can be discovered, and if there is no conflict with other IP-addresses on the LAN, the IP-address of a PC might be temporarily changed to be within the subnet that the ZipZerver can address, and then the admin web page used to fix the address. It may be necessary to use a keyboard and monitor on the ZipZerver, and run a console command, after login as userid "admin". See section Useful Command-line Operations for admin for more details.
Server for Web BrowserUpdated 2019-Nov-26 for Initial PageThe Help for the admin web page with the link Network -> Information has a general discussion about Enable for multiple protocols. Since all administration for a ZipZerver is done through web pages, the Server for Web Browser cannot be disabled, but you can select http-only, https-only, or some variants of both. The default is to Enable both http and https, but secure-https does not run until --
Note that if you select Enable secure https only before the other necessary steps, http will continue to run and it will appear that nothing has happened. However, when the last step is done, https will start and http will no longer respond. If everything was ready, the response will be delivered on that same connection and will be visible, but any further use of http will get no response. The choice Enable both, redirect admin pages to https is intended to protect the admin password (and passwords of other users who have admin privileges). After the redirect, any links that are selected will be to encrypted pages. The choice to redirect all pages to https will also cover http access to share-data. (An exception -- access to a share or directory named /.well-known/ is left as http, not redirected to https, since it is used by ACME and perhaps other Certificate Verification protocols. See Help for web page https Certificates.) A connection made to a server program on any network node uses a port number, a value from 1 to 65535, with particular network services usually found at a particular port number. The server program for unencrypted web pages is generally found on port 80, and for https encrypted web pages is 443. There are times when an alternate port can be used, such as when multiple ZipZerver must share a single external IP address behind a NAT-Network-Address-Translation firewall-router. The fields Web Port Number and https Port Number can be used to set such alternate port numbers. After changing the port number and hitting the Save and Apply Now submit button, the result message will be delivered using the current connection with the current port, and thus will be visible. Upon success, such as changing from the default web port of 80 to 82, subsequent web requests would use a form such as -- http://ip.ad.re.ss:82/admin/netinfo or for simple share-data access -- http://ip.ad.re.ss:82 in which ip.ad.re.ss is IPv4 address or a DNS name. For IPv6 addresses, the syntax is -- http://[ipv6:addr:ess:in:hex]:82 For the admin web pages, a new login may be needed. The field https Server Name, mentioned above, is part of the configuration for the https secure server, and should be the primary name that a Certificate would certify. If the Certificate will be for a name that is part of DNS, such as www.littlesystem.example.com, put that here. If the Certificate Authority is very local, for names defined by a local Name Server, such as littlesystem.local, use that. If connections will be made by explicit IPv4 or IPv6 addresses, enter one of those. The Help text for the https Certificates web page has more to say about additional SAN-Subject-Alternative-Names, either DNS format or IP format, that can be used for a secure connection. The selection for Initial Page Share/path is intended to provide an alternative to the standard list-of-shares initial web-page. The value List of Shares will select that default behaviour. If another share is selected, then
For web browsing, for a ZipZerver named NodeName joined to an A-D Active-Directory domain named DOMAINNAME, a domain userid can be used where a local zerveruser can be, in various forms.
For secure-https, the DOMAINNAME\\userid variant, with two "\\"-slant characters, can also be used.
The full discussion of the form of userids needed for each protocol for both A-D Active Directory and for older NT-4-type domains is given with the help text Domain userids, Local userids, and Protocols
Manage https CertificatesUpdated 2022-May-27 for more ACME-CA and data fieldsWhat is a Certificate, and why would I want one? Public-Key cryptography, in which a user creates a key-pair, a Private-Key that remains secret forever and an associated Public-Key that can be freely distributed, has two uses -- (1) message secrecy, in which a message encrypted by anyone with the Public-Key can only be decrypted by using the corresponding Private-Key, and (2) digital signature, in which a message (or message digest) is processed using the Private-Key producing a value which can be verified by anyone with a copy of the Public-Key, that the message must have been signed by the holder of the corresponding Private-Key. The most common public-key method is RSA, in which two large prime numbers (e.g. 1024 bits) are found. The Public-Key is the (2048 bit) product and a small exponent, the Private-Key (the inverse of the small exponent) is found with knowledge of those two primes, and the Private-Key cannot be found from the Public-Key as long as it is infeasable to find the prime factors of a number that large. Web connections using the https protocol and SSL/TLS encryption (rather than http) are encrypted using public-key mechanisms that can be numerically equivalent to, but philosophically distinct from, ssh-Secure-Shell. For example, both can use RSA public-key encryption, and might use the same numeric values in some circumstance, but differ on the central issue in Public-Key cryptography -- which Public-Keys are valid and for whom? The problem is deciding which Public-Key actually belongs to a potential communication partner. There could be dozens of Public-Keys floating around the internet labeled as "Bank of America", some current, some obsolete, and some created by crooks. The solution used for https SSL/TLS encryption is to use CA-Certificate-Authorities, which are entities that will verify that a Public-Key is associated with something, perhaps an Organization Name (such as Bank of America), perhaps just a DNS-Domain-Name-System value (such as apod.nasa.gov, the Astronomy Picture Of the Day, as certified by Lets Encrypt). A CSR-Certificate-Signing-Request has a digital signature that verifies that the Requester holds the Private-Key for a given Public-Key, and is sent to a suitable CA which may be able to verify the DNS values by automatic probe, and may verify the Organization and Location claims by other means. If satisfied (and if suitable payment occurs), the CA creates a Certificate, a digital signature that ties the new Public-Key with the identity claims made in the CSR to the Private-Key (and the reputation) of the CA. A browser that supports https will have the Public-Keys of some number of CA which are implicitly trusted to verify other Public Keys. When you go to a website using https, that website presents a Certificate as part of the connection setup, and the browser can cryptographically verify that the Certificate was signed by a trusted CA (or was signed by a Key that itself was signed by a trusted CA, etc). When there is not such a Certificate, or the cryptographic checks fail, or (more likely) because an arbitrary validity interval has expired for one of the Certificates in the chain, the browser presents an alarming-looking message intended to convey that the website that you got may not be the website you wanted, including the text "A legitimate bank will not ask you to over-ride this warning." The obvious weakness begins with the phrase "implicitly trusted." A recent Firefox browser has many dozens of CA listed (under Edit -> Preferences -> Privacy-Security -> Certificates -> View Certificates), with a number of reassuring-sounding Organization Names identified with the US or various places around the world, including -- China (CN), Estonia (EE), Hungary (HU), Luxembourg (LU), Norway (NO), Romania (RO), Slovakia (SL), and Turkey (TR). There has been at least one instance of a successful attack on a CA, DigiNotar, a Dutch company (now defunct), with rogue certificates created, including for *.google.com, apparently targeting gmail users in Iran. Still, the mechanism works reasonably well in practice, protecting millions of connections every day, and allowing any user anywhere to connect to any web-site without requiring any advance preparation. There are alternative ways of judging the validity of Public Keys. When ssh-Secure-Shell users first encounter a new Public-Key value, they are presented with a long character string that represents a cryptographic hash of the public key, and asked to verify that the key is as expected. Under some conditions, such a user might use an out-of-band communication with the key owner to verify the full hash value. However, ssh-Secure-Shell users in general (and ZipZerver use for Encrypted SmartMirror and Encrypted FTP, in particular), are likely to interact with a relative handful of correspondents (compared with the millions of possible https websites). That said, the ssh-Secure-Shell code has a limited @cert-auth capability, which ZipZerver does not currently use, and which does not obligate anyone to accept any ssh Public-Key. Another Public-Key management choice was created by the original PGP-Pretty-Good-Privacy crypto program. Rather than a single digital signature from one CA, a PGP Public-Key file might contain any number of signatures from any number of individuals, each asserting that the individual is confident that the Private-Key is held by the identified owner. No one is obligated to do anything with these signatures, but if you did trust the judgement of one or more of those signers, cryptographically verifiable if you have their Public-Keys on your keyring, you might choose to trust the identity of that key. Or not. This is described as a "web of trust", and can work pretty well if the number of correspondents is a handful or dozens, and gets pretty unworkable if there are a thousand people on your list. ("Three may keep a secret, if two of them are dead." -- Benjamin Franklin) Thus a Certificate issued to your web-server from a widely-accepted CA-Certificate-Authority allows arbitrary visitors from anywhere to access your website securely, with some confidence that the website they got was the one intended, without intermediaries being able to observe the content or modify it. (ISP-Internet-Service-Providers that add yet-another pop-up ad to every un-encrypted webpage find it inconvenient when everything is encrypted.) Do I absolutely have to deal with a Certificate Authority to use https? In the past, one could use https for security without dealing with a CA-Certificate-Authority by creating a Self-Signed Certificate, in which the digital signature is made by the Private-part of the Public-Key. This has been suitable for testing and development, is fine for message secrecy, but does not re-assure a visitor about the destination reached. In the past, an https connection made to a website that presented a Self-Signed Certificate would trigger warning messages to the browser user, similar to an invalid or expired Certificate, and sometimes includes a snarky reference to being Self-Signed. One or more browsers, including Firefox as of 2022-May, now present a message about an unsuitable key, with text that there is no over-ride for this condition. Although well-meaning, and perhaps justified by the wide availability of free certificates (as discussed below), this change does complicate testing and development. If you have any systems which can run program openssl and an ability to decode instructions found on the internet, you can also create your own little CA-Certificate-Authority for use within a limited LAN, in which a Certificate-Authority master Certificate is created and then imported into the set of "implicitly trusted" CA for some number of local systems and browsers. This can be useful for testing, and may be desirable if some https servers must be identified by IP address or local name (rather than by a DNS name). Unless there are careful controls on access to the Private-part of the little CA, this is undesirable since bogus Certificates can be created that would be accepted without complaint by a user at a system which had imported that master Certificate. It is sort of like playing with fire. If I want a widely-accepted Certificate, do I absolutely have to pay for it? There are many different CA-Certificate-Authorities with different policies about what they will sign. Some offer "EV-Extended-Validation" service, in which they will verify the Organization Name, Location, and Address along with the web-site DNS value. If you are a bank, you might want one of those, so that when a web-site visitor clicks on the little lock icon, there is some re-assurance about the identity. LetsEncrypt.org "is a free, automatic, and open Certificate Authority," and does offer limited Certificates for free "because we want to create a more secure and privacy-respecting Web." The only thing that is certified is the DNS value (or values -- the Astronomy Picture of the Day web site has a Certificate for around 60 sub-domains), with nothing about the Organization or its location. This can be done for free by being completely automated. A protocol named ACME-Automatic-Certificate-Management-Environment is used to automate the steps. The verification involves a "challenge" of the form, "If you really do control domain (domain.name.for.the.cert), then put a file named (long unlikely name) with contents of (long implausible contents) at this point in the web-site space." A DNS lookup is then done for the IP address, and http port 80 (not https port 443) is used to look for that file. This is all automated and takes only a few seconds. This means that a Certificate from LetsEncrypt.org will be for a name that is part of the world-wide Domain-Name-System, and cannot be a name that is only known to a LAN-level name server. Also that the target system can be reached on port 80, and thus cannot be connected by a home-oriented ISP that does not allow incoming port 80 requests (but see below for a clumsy, partial work-around). There are some other challenges that can satisfy LetsEncrypt.org but that are not directly usable from a ZipZerver. The effort for free Certificates is being done by people that believe in Open Standards, with the intention that ACME will become a standardized protocol that could be used by other organizations to offer automatic Certificates. Several alternatives to LetsEncrypt.org are included with the ACME-protocol code (as of 2022-May), as described below. A few choices for key management are listed as ACME actions, and can (theoretically) be done with any of these ACME-CA. So what do you actually see on the admin web page for https Certificates? The upper part of the page may show the result of running the most-recently selected key-management action. There may be useful warning or error messages that approach being understandable. After the optional results, the first section shows the current status of the https keys. If a Public-Private key pair has been created, the file timestamp for the Private Key is shown. Any Public Certificate will have more data to show, being Public and all. This file may be the result of upload from a CA-Certificate-Authority (by one of the operations described below), and may have a time-stamp very different from the file for the Private Key. As the result of an upload, it is possible for the wrong file to be in place, and so a status line shows if the Certificate does or does not correspond to the Private Key. The subject line contains most of the identity information of the Certificate, with the issuer line showing the identity of the signer. More can be said about these fields below. The other Certificate information pieces relate to the signing procedure. If a CSR-Cert-Sign-Req exists, its info is shown next, starting with the file timestamp and an indication whether it does or does not correspond to the Private Key. The CSR subject is the direct result of the data fields below and a CSR-create operation. The purpose of a CSR is to be sent to a CA. The link Show CSR as Text is intended for download as a file or copy-and-paste as text as a step to accomplish that. The text is in PEM (Privacy-Enhanced-Mail) format for easy handling. For the curious, there is also a link to Show CSR with fields interpreted. There are a lot fewer reasons to use the download link Show Certificate as Text, since this file is uploaded from a CA. Perhaps if a good Certificate is loaded on the wrong machine, it might be useful to download this file. For the curious, however, there is even more to see with the link Show Certificate with fields interpreted. The Certificate and CSR files are generally readable, unlike the Private Key file which is set accessible to (part of) the web server program, but with no permissions in general, including the user of the admin web page. The purpose of the link Show Private-Key as Text is to fail, thus demonstrating that the permissions and ownership of the Private Key file have been set correctly. The Server for https Web may show "started" if there is a Private Key, there is a Public Certificate that corresponds to it, and a suitable entry for https Server Name on the Network -> Web admin web page, and maybe some other conditions. Yes, it is unfortunate that the https Server Name is on another web-page. The ACME auto-renew line shows if a daily task is scheduled to check if an ACME Certificate is about to expire and needs an automatic renewal. If you are using a conventional CA (other than ACME), this should be something other than "yes", and if so, you should run the ACME Drop Auto-Renew operation listed below. Identity Data for a CSR-Cert-Sign-Request and Certificate The actual data elements of this form are used to create a DN-Distinguished-Name, which will be shown as part of a CSR or Certificate as subject, with Country (C), State (ST), Location (L), Organization Name (O), and Organization Unit (OU) fields identified with little tags. The Country uses two-letter country identifiers. The Common Name (CN) is also shown as part of the DN-subject, and should have (one of) the DNS value of the server. It and any SAN-DNS (and sometimes SAN-IP) values have direct impact on the effect of a Certificate. If a browser-user uses https with some name to reach a web-site, a Certificate is presented, cryptographic checks are made, and the Common Name and set of SAN-Subject-Alternative-Name (DNS) fields will be checked for the name that was used to connect. If not found, the warning about possible wrong website is given. If a connection is made by IPv4 or IPv6 Address directly then the Common Name and SAN-Subject-Alternative-Name (IP) fields will be checked for that value. The Common Name field will be initialized with the https Server Name from the Network -> Web page. Multiple SAN-DNS and SAN-IP values are allowed. There is a data entry field for one of each, and if it has data during an operation, including the No operation needed at the bottom of the operation list, then another data field is offered. The requirements for the DN-subject fields are somewhat vague. Different CA may have different requirements. (It looks like the LetsEncrypt.org CA only requires, and only uses, Common Name and SAN-DNS values.) Some syntax checks are done on the SAN-DNS and SAN-IP fields, but those are given as a Note, rather than as Error or Warning, due to occasional odd browser behavior. A test with several SAN-DNS values was rejected by a Firefox version for the Common Name, which apparently needed to be duplicated among the SAN-DNS set. It would connect using IP values listed under SAN-IP, but lynx (a text-based browser useful in test scripts) would accept the IP addresses only when also listed with SAN-DNS. That is a wonderful thing about standards -- many things can be made to work provided you have a hammer and can pound enough things until they fit. The Email Address field is included based on a template from the openssl program, but is only used as part of an initial ACME CSR operation. Valid Days is only used when creating a Self-Signed Certificate. The validity interval in a Certificate from a CA is determined by the CA. So what exactly do I have to do to get a Certificate? The set of key-management operations is given next. There are two ways to generate a Private Key (one with a Self-Signed Certificate, one with a CSR-Cert-Sign-Request), two ways to get a CSR (one while generating a Private Key, one using a previously-created Private Key). When a CSR has been created, it can be retreived using the Show CSR as Text link above with the CSR-Cert-Sign-Req information, or further below in a Copy-and-Paste text area labeled CSR or Certificate. The CSR is to be sent to the CA-Certificate-Authority of your choice, who should respond after a time with a Certificate. Such a Certificate can be loaded as Copy-and-Paste text in that same text area, using the Load Certificate from text-area operation, or loaded as a file with the Filename selector and the Load Certificate from file operation. If you use the copy-and-paste text area, be sure the CERTIFICATE REQUEST data is not left over in the CSR or Certtificate area before pasting in the CERTIFICATE text. The Fill-in data fields... would be used to fill-in the data-fields for editing or additions before creating a new CSR. The quickest and easiest way to get a widely-accepted https Certificate is to Create New Key and a CSR followed by ACME Submit a CSR (for those systems for which an ACME Certificate is suitable, from LetsEncrypt.org or one of the other ACME-CA). Otherwise, the Create New Key and Self-Signed Certificate will get you on the air in one easy step, at the expense and aggravation of clicking through warning messages on each client system (assuming that your browser will still allow that). That one step can be followed, however, with the operation For an existing Key, create a CSR which can be sent to the CA of your choice. What has to be done to get an ACME LetsEncrypt.org free Certificate? As described elsewhere in these notes, you can get a free Certificate from LetsEncrypt.org or another ACME-CA-Certificate-Authority, provided the name or names that you want are DNS values that resolve to a system (or systems) under your control (rather than LAN-local names or IP Addresses), that a Certificate with only those names is suitable (no Location, no Organization), and that http port-80 is open to the general internet for use in the file-presence challenge (there can be a firewall and/or port-forwarding router, but the challenge uses http on port-80, rather than an alternate port number). For identity data, fill-in one of the DNS values as the Common Name, and any other DNS values as SAN-Subject-Alternative-Name (DNS), using the No operation needed (non-)operation to get additional data fields (e.g. polycosmic.example.com and www.polycosmic.example.com). Fill in the Email Address. All the other identity fields can be left blank, and will be ignored in any case. Select operation Create New Key and a CSR-Certificate-Signing-Request. Before the next (and likely final) step, ACME Submit a CSR-Cert-Sign-Req, select the ACME-CA-Certificate-Authority to use. Several are listed, all running V2 of the ACME protocol (and previous versions which could select V1 should not be used). One of these choices is for EC-Eliptic-Curve (rather than RSA), which ZipZerver is not yet ready to use. Some of these are for testing (more about rate-limiting below). The data field can be used to specify another server if that is necessary or desirable (perhaps for another ACME provider, and was essential during testing when an internal staging server was needed). More can be said about the choices for ACME-CA, and other data fields, below. The ACME client and ACME-CA server may take several seconds (to run the file presence challenge for each DNS value), return the Certificate, install it, and signal the web-server to begin using it. The only painful part of the whole operation is having to read all of this commentary. If things go wrong (e.g. polycosmic.example.com works fine, but the appropriate name server for www.polycosmic.example.com returned an obsolete IP or no IP), then perhaps the test version of the ACME-CA service should be used to work out the problem. The full service LetsEncrypt.org server (and perhaps the other ACME-CA) is rate limited in a number of ways "to ensure fair usage by as many people as possible," including Certificates per Registered Domain (20/week), and Failed Validation (5 failures per hostname per hour). If there is a problem to be worked, the test/staging server can be used multiple times with much higher limits. The result of successful use of the test/staging server looks like a Certificate (and the https web-server will start), but the digital signature is from a testing key, rather than the widely-accepted CA key. After the problem has been worked out, repeat the ACME Submit a CSR operation using the full-strength server. On success (full or partial), a daily task will be scheduled to check if renewal should be done. This will be shown above with the status information line ACME auto-renew as "yes" with an entry for Next sched hour:min, likely some random minute after local midnight (to spread out the load on the ACME-CA servers). The ACME Server to be used for renewal will be shown. The task will run every day and check the Certificate. If the expire time is within 30 days, an auto renewal will be attempted and this may involve another "challenge" with a file name and contents that are checked for each DNS value, using access by http (not https) through port-80. If the auto-renew check has not been running or has failed for some reason, ACME Renew a Certificate can be selected, which will attempt the renewal immediately, and make sure that the auto-renewal check task is scheduled. If a Certificate is obtained from another CA (perhaps to define the Location or Organization fields), be sure to run ACME Drop Auto-Renew operation, so that the auto-renewal-check task does not try to get a renewal Certificate from the ACME LetsEncrypt.org server (which would probably work, in the sense that a Certificate would be created and installed, but that ACME Certificate would lose the Location and Organization entries). List of ACME-CA providers and other data fields As of 2022-May, an updated version of the code which runs the ACME protocol is being used (acme.sh-3.0.5) which lists a number of ACME-CA that can be used. LetsEncrypt is still in the list (of course), but others are provided. A table and short summary of ACME-CA-Certificate-Authorities known to (and supported by?) the code which handles the ACME protocol can be found at https://github.com/acmesh-official/acme.sh/wiki/CA along with short discussions of other topics.
In this context, "works" means that a Certificate was issued. Testing of automatic renew has been done with LetsEncrypt. The difficulties with different ACME-CA listed above are very likely due to the ZipZerver environment in which an "easy-to-use" (hah) interface wraps around the ACME protocol code, rather than providing a command-line interface directly. It is a good thing to have alternatives to LetsEncrypt.org, even when it seems like the US airline and medical insurance model is the inspiration to make otherwise simple things incredibly complex, unpleasant, fragile and expensive. The invisible hand of the free market might actually create a product or service worth paying for. Because of references in SSL.COM emails and Google error messages, there are fields for entering values for eab_hmac_key and for eab_kid, which would be used in linking an email address to an account at one of the ACME-CA. If both values are present, an attempt to register-account will be done. So far, this has not been successful. The field --valid-to is intended to allow easier testing of automatic renew. According to the docs, a value such as yyyy-mm-ddThh:mm:ssZ would request a very particular ending date-time, consistent with the issue interval for the ACME-CA. A value such as "+32d" is supposed to request an ending in 32-days, which will make it quicker to test that the daily check for 30-days to expiration will refrain from renewal at some tests, and will attempt renewal at others, without needing 60 or more days to run each test. The selection Require proper, current web-certificate from ACME-CA is the default, and will help assure that the web-site claiming to be e.g. LetsEncrypt.org is the real web-site, and not an imposter. This is strongly recommended. The alternative, Do not require, is provided for a pragmatic reason. The firmware running this ZipZerver contains a number of "root" Certificates intended to verify Certficates presented by web-sites such as LetsEncrypt.org. These root Certificates can expire, and so a valid Cert from LetsEncrypt.org can be rejected, based on the built-in, available Certs. If validity is required, attempts at renew (or new cert issue) fail. This is not theoretical. A test system running ZipZerver Release 1.2.06 was failing to renew a LetsEncrypt.org Certificate, due to a set of Root Certificates taken from a Slackware release from several years earlier. It would be handy, of course, if an error-message of the form "LetsEncrypt.org cannot be verified due to expire of the following intermediate cert..." This, of course, did not occur. The renew-attempt would just fail. No hint. The next release, 1.2.07, had an updated set of Root Certificates, and then the LetsEncrypt.org renew was successful.
kernel/entropy_avail Since effective cryptography is important to so much of system communication, and since a good source of random or pseudo-random values is critical to cryptography, the linux kernel attempts to keep track of entropy available, generated by the unpredictable timing details of communication messages, physical devices like disc drives, and human interaction with a mouse or keyboard. There are well-defined interfaces for access -- /dev/random is to be the source of random values, and a reader-program should be suspended until sufficient is available, /dev/urandom may be random or pseudo-random (generated in a way that will pass statistical tests for random, but which might be predictable given sufficient knowledge of internal values), which should provide material to a reader program without suspending. There are hardware devices that purport to create random values based on analog processes such as heat generation. The Linux kernel configured in ZipZerver will use the results of such devices, if available, but does not depend on them. If there is a hidden pattern known to some TLA-Three-Letter-Agency, those bits will be mixed in with bits from random timing events. Limits on entropy in current computer hardware --
The entropy_avail is the Linux kernel's indication of how much random-ness is available. Is it measured in bits? Discussions of entropy tend to be hard to follow. This value is displayed with the idea that if 2048 bits or more of entropy are available, then creating a 2048-bit RSA key should be high-strength random and very secure. It was expected that after that many bits of entropy were used, that the result value would be a lot smaller. That does not seem to occur, reasons not understood. This field provided after reading about a study of many, many RSA keys, a non-trivial number of which were crackable, in which a prime factor happened to have been used in two or more different keys. This seemed to be the result of some number of small ("embedded") devices in limited applications which, upon first boot-up, would generate an RSA key with the available entropy, whcih was very limited -- current date and time (perhaps to the microsecond), the unique ID in the communication hardware (which is not a secret), and not much else. The resulting keys, this study claimed, were of poor quality. How to get an ACME Certificate behind an ISP Firewall that blocks port 80 If a ZipZerver is located in a setting behind an ISP-Internet-Service-Provider that blocks incoming port-80 (for http), but allows incoming port-443 (for https), it may be possible to receive and install an ACME Certificate. (Even if possible, it should be noted up front that automatic renew will not work very well.)
Among the things that can to wrong with this Certificate method -- from time to time the IP-address for your router's external link can change. If the DNS-service was defined with an IP-address, that will go nowhere or be directed to that IP-address now at a different location. You will have to update the definition at the DNS-service. (These details were first worked out after a home router from NetGear offered a free service for a name within ".mynetgear.com", with an update to the DNS-service, noip.com, whenever that IP address changed, automatically by the router. Thus the mapping to that name would always stay current.)
Static Web Page FeaturesNew 2019-Dec-26When http or https are used to access any directory found on a ZipZerver which contains a file named either index.html or index.shtml, a directory listing (the default behavior) is not produced. Instead the contents of that index file is sent to the browser directly. That file can contain any text and web-page formatting, may refer to any number of image files (or other media files), and may contain clickable-links to other files within that directory, or directories on the system, or anywhere else to which a clickable-link may point. The result is a set of web-pages that are (almost) completely static and unchanging and may be of any size, perhaps arbitrarily large. The contents would change when one or more of the files are updated with new content, or (in a much more limited way) by the SSI-Server-Side-Include mechanisms described below. Such a set of web-pages would not work for a transaction, in which details of a series of customer choices must be handled. However, it might be suitable for a catalog of products with description, images, and price info, or reference material which has infrequent updates. If the Network -> Web page item for Initial Page Share/path is set to select a share, with or without a sub-directory within that share, then that directory or sub-directory will be checked upon initial http or https connection to the system.
Such a share, if it contains reference-like material, --
Each of the web-page files must have at least minimal form -- <html> <head> <title> Page Title Text Goes Here </title> </head> <body> html-formating and web-page text goes here. </body> </html> SSI-Server-Side-Includes for limited variation in the static text SSI-Server-Side-Includes are a mechanism that can be used with the (almost completely) static web-pages. There is a tutorial on the apache web-site (which is the project that creates the web-server) at httpd.apache.org/docs/2.4/howto/ssi.html Server Side Includes, and an apparently more complete discussion associated with the "include" module at httpd.apache.org/docs/2.4/mod/mod_include.html Include Module. Various points about the use of SSI-Server-Side-Includes with this system --
Server for Windows SMB/CIFS ProtocolUpdated 2023-July-08 to drop AD_DC and DOM_PDC choicesThe Help for the admin web page with the link Network -> Information has a general discussion about Enable for multiple protocols. A crucial choice for any machine running Windows SMB/CIFS Protocol is membership in an "Active Directory Domain." Several choices are available. Stand-alone server is the simplest choice. Machines with the same Workgroup name can easily locate shared resources, printers and directories, within that group. Shared resources on machines in Workgroups with other names can also be found and used. For such an arrangement, there is no single centralized point of control, and the userids, passwords, permissions and restrictions are done on an informal basis. For use within a Workgroup, very little else is needed beyond the name of the Workgroup. Other entries on this admin web page related to Domain values can be ignored. Join an Active-Directory realm, in contrast, provides elaborate possibilities for centralized control. Once that is done, the userids known to that Domain, as part of that Domain or as part of another "trusted" Domain, are also known to the ZipZerver for use by SMB/CIFS protocol and can be used with most of the other protocols. Share permission can be allocated to such "DOMAINNAME\Userids" using ZipZerver tools (however using domain tools to control access to ZipZerver resources is not supported). Upon attempted access, userid and password checking is done by Domain services, not by the ZipZerver, so that a password change in one place, with the Domain, will be reflected in accesses to any shared resources anywhere within the Domain. Since Active-Directory services depend on a DNS Domain-Name-System server that understands all of the Active-Directory specialized names, it may be necessary to add the static address of a D-C Domain-Controller to the set of DNS servers on the Network -> TCP/IP web page. Join an NT-type workgroup domain is still supported, which is the operation provided by earlier versions of this firmware. If there is such a domain, the DOMAINNAME\Userids known to the PDC-Primary-Domain-Controller can be used on the ZipZerver, with userid and password checking done by Domain services, as with Active-Directory, for SMB/CIFS protocol and most of the other protocols. Some of the network mechanisms are simpler, so on a single LAN segment, an NT-type domain can run without help from DNS. It turns out that samba can Run a (limited) NT-type domain PDC-Primary-Domain-Controller without too much trouble, which is not necessarily a good idea. However, it does mean that testing of the operation to join an NT-type domain can be done without having to raid the Smithsonian for a suitably ancient NT-server. The local userids defined on the ZipZerver are available to other systems which join the NT-type domain. Note that one of those local userids must be created in advance for any system which intends such a join -- Before a system named "GUMDROP" can join, the ZipZerver must create a userid named "GUMDROP$". This is in contrast to an A-D domain, in which a Join can occur without any preliminary step on the domain controller. Some versions of the firmware (most recently Version 1.2.10) can Run a (limited) Active-Directory domain controller in which local userids will be available to other systems as DOMAINNAME\Userid. This is primarily intended for testing. A lot of programs need to be added to the firmware for this option that are otherwise unneeded, so some versions will have an error for this choice. Some factors involved with using this choice are given with the help text here. Perhaps Windows SMB/CIFS protocol should be off, if there are no Windows clients for the ZipZerver. The choice Windows SMB/CIFS protocol should be off and previous SMB/CIFS state data should be discarded is provided as a transition mechanism before attempting to Run a (limited) Active-Directory Domain Controller. It might also be used for some kinds of very serious protocol problems. It is hoped that this would never be needed outside of a testing environment, and discards, among many things, the file with the SMB/CIFS passwords for ZipZerver local userids. After this step, local userids would still work for all of the other protocols, but local userids would not work for SMB/CIFS until each local userid was re-created with a new password definition. In general, a reset SMB/CIFS operation would use the protocol should be off operation and then back on (e.g. Stand-alone or re-Join an A-D realm) but should not use the state-data ... discarded operation. Transitions from one mode to another are not expected to be frequent, but are allowed. If the presence of too many DOMAINNAME\Userid is causing a problem after join to an Active-Directory Realm, it is legal to start operating as a stand-alone server. (Earlier versions of this firmware had a config choice to either use DOMAINNAME\Userids or not, in anticipation of this kind of problem.) The Windows Workgroup/Domain Name field provides the Workgroup name for a Stand-alone server or the Domain Name for an NT-type domain. For Active-Directory, this field must not conflist with the Active Directory Realm name (and if blank, for Active Directory, will be assumed as the first component of the DNS value of Realm). The field Active Directory Realm is only needed to join (or to run) an Active-Directory domain. It will be FQDN-Fully-Qualified-Domain-Name, a name in DNS format such as "polycosmic.mydomain.com" or "thisdom.local" which will be resolved by the DNS Name Servers configured on the Network -> TCP/IP admin web page and/or provided by DHCP. This name must resolve to the system which is running Kerberos for the Active Directory Domain. The Realm field can be left blank if the ZipZerver will be a stand-alone server or part of an older NT-type domain which uses NETBIOS mechanism with the Workgroup name to find domain resources. The process of Joining an Active Directory Domain requires a Domain User for Join, a userid from that Domain with sufficient privilege for the Join. In some cases, this does not have to be "Administrator". (If the domain-controller is another ZipZerver, the userid with enough privilege should be "admin", using the password for admin on that other ZipZerver.) This userid and associated Password are used during the Join operation, the transition from something else to either Join an Active-Directory Realm or Join an NT-type workgroup domain. Earlier versions of this firmware needed to save this userid and password for use with fetching the list of userid and for re-join to a domain after reboot (with interesting symptoms if that userid later changed the associated password at the domain). The current code does not need this mechanism, and the password will be discarded after the join. The result of a Join is not setting a flag-bit for "join", but the establishment of a shared-secret value between the machine and the Domain, so that messages about userids and passwords, and the identity of the machines requesting and responding, can be cryptographically authenticated. There seem to be a large number of reasons why the Join-Domain operation might be rejected, including unrecognized Domain User, wrong password, or insufficient privilege. One that occurs regularly during system testing is because a machine with that name is already a member of the Domain. Here is a machine claiming to be "TEST1", but the message offered shows no evidence of knowledge of the shared-secret value known to the first machine named "TEST1", so the Join is rejected as coming from an imposter. The message is not over-eloquent -- "Unable to join domain DOMAINNAME". The fix is to perform the Domain operation to select and delete the previous instance of the machine name from the Domain at the Primary Domain Controller, and try the Join again. After a Join operation, the Windows Domain status will show OK -- Member -- DOMAINNAME and the Security -> Share-User-Permission admin web page will show DOMAINNAME\Users as part of the list of Users that can be granted permission to a Share. The Windows Domain status field can show various other values if a domain join has never been done (no domain access), or for various failures, such as domain controller not accessible or various required programs not properly running. After a successful Domain Join, a restart of the ZipZerver is not required, however, if there were already SMB/CIFS client connections when the Join was done, there can be odd Share-permission and Share-denial problems. After Join to a Domain, it appears that a machine can be a member of one and only one Domain. The machine could join to a different Domain (with potentially interesting results for SMB/CIFS clients whose userid and password values were checked by the former Domain). Thus ends the list of admin web page controls that are related to the Domain Name field. The selection for Minimum SMB Protocol Allowed can prevent ancient versions of Windows from negotiating the parameters of a connection, and can thus be a mitigating factor in the presence of malware and ransom-ware that exploit weaknesses in some of the early protocols. This selection has been added, in part, due to the enthusiasm expressed at the web site Internet Storm Center, oriented to computer-security and incident-responders, about a new version of Samba that will have SMB2 as the default for server minimum protocol. If you do not make a selection, this aspect of your configuration will be unchanged. However, the highest protocol level that works in your environment seems like a good idea, and may reveal the presence of an XP system in some forgotten corner. The description of which SMB protocol versions correspond with which Windows versions is taken from the Samba documentation, and MSFT might have a different interpretation. If you have other network appliances that use Samba to make client connections, you may have to experiment to discover the minimum protocol that is workable. The field Syslog Message Level is used to change the amount of commentary produced by the SMB/CIFS Server as viewable at the Status -> Current Details web page. The default value of "1" reports only major events or conditions. Higher values, up to "10", can produce a lot of syslog messages, with the effect that the most recent messages retained and shown may cover only the most recent minutes or seconds. An elaborate syntax is available, used by the developers of the Samba code, for selective logging. A value such as "1 winbind:3" uses a lower log level for most of the code, but a higher log level within one or more sections. When this value is changed and applied, new SMB/CIFS connections use the new value immediately, and existing SMB/CIFS programs are signaled so that the new value takes effect within a minute or so. There are applications which can access a ZipZerver Share using a UNC-style connection, rather than mapping a drive, which might spend otherwise idle time by initiating access and then ending access to the Share. Each of these operations can create an entry in the Status -> Login History for Begin-Share-Use and End-Share-Use. Sometimes these are the messages that are needed to help diagnose a problem, especially if it is for a "Public" Share in which there is no "Login" or "Logout" messages. However, if these messages are incessant and useless, the field Share Name Begin-use, End-use with default value Should be written to Login History Now can be set to value Should not. This configuration choice operates differently from almost all the others. Generally, values can be Saved with configuration data but do not take effect until an Apply Now operation (or system restart) is done. The effect for Share Name Begin-use, End-use field occurs as soon as Save is done, regardless of whether Apply is done. Domain Userids for Windows Protocol For Windows protocol, a ZipZerver named NodeName joined to an A-D Active-Directory domain named DOMAINNAME, a domain userid can be used where a local zerveruser can be, in various forms.
In some circumstances, it might be possible or desirable to login to Samba using "nodename\zervuser" or "NODENAME\zervuser", rather than just the local userid "zervuser" on the ZipZerver named NODENAME. Also, under some conditions it may work to use "\userid", "\\userid" or even just "userid" (when there is no conflict between a local userid and a domainname\userid). The Samba protocol handler may recognize these forms and be able to verify a password. The ZipZerver does not allocate share permissions to these forms, however, so these might only be useful for access to a public share. The full discussion of the form of userids needed for each protocol for both A-D Active Directory and for older NT-4-type domains is given with the help text Domain userids, Local userids, and Protocols
Domain userids, Local userids, and ProtocolsUpdated 2019-Mar-12Because each protocol server interacts (or fails to interact, for SmartMirror-rsync) with the A-D userids in its own special way, the chart of workable combinations of domainname, DomainName, or DOMAINNAME, which separator, "\" or "\\", and which form username or UserName, is irregular and somewhat displeasing. The summary for this chart -- Basic SmartMirror-rsync cannot use any DOMAINNAME\userids (but encrypted SmartMirror, rsync_ssh, can). The other protocols will accept DOMAINNAME\userids, however basic FTP needs DOMAINNAME\\userids, with two "\\"-slant chars. In this chart, the entries of local-userid zervuser and DOMAIN\userid marked with "Yes" work with both A-D Active-Directory and with an older NT-4 style domain. Those marked with "Yes*" work with A-D Active-Directory, but do not work with the older NT-4 style domain.
"Yes*" -- entry works with A-D Active-Directory, but does not work with older NT-4 style domain. In some circumstances, it might be possible or desirable to login to Samba using "nodename\zervuser" or "NODENAME\zervuser", rather than just the local userid "zervuser" on the ZipZerver named NODENAME. Also, under some conditions it may work to use "\userid", "\\userid" or even just "userid" (when there is no conflict between a local userid and a domainname\userid). The Samba protocol handler may recognize these forms and be able to verify a password. The ZipZerver does not allocate share permissions to these forms, however, so these might only be useful for access to a public share. For those creating command-line scripts to use the ZipZerver protocols, the following examples are offered. Note that for a typical Linux or Unix command-line interpreter such as bash, the "\"-back-slant character is used as an escape to force the following character to be interpreted in a non-usual way. For example, a "\"-slant character at the end of a line (just before the New-Line character), forces that New-Line character to be ignored, so that the subsequent text (after the "\New-Line" pair) will be considered part of the same line. This means that if the value for a DOMAINNAME\userid has a single "\"-slant, the command-line representation has two, in which the first forces a non-usual interpretation of the second, namely as a literal "\"-slant character -- "DOMAINNAME\\userid" Where the chart calls for value DOMAINNAME\\userid, with two "\\"-slants, the command-line representation uses four -- "DOMAINNAME\\\\userid"
The following code examples attempt to fetch a file
whose name is given in DMN_MARK_FILE
from a directory named /dmntest/ within a share SHARENA
using a user given as TESTU with a password of TESTP.
Except where noted below (mainly basic FTP),
the TESTU value is of the form DOMAINNAME\userid (with one "\"-slant),
and must be set for a typical Linux/Unix command shell with --
Program ssh, used directly and indirectly for rsync_ssh and ftp_ssh, strives mightily to ensure that an interactive passphrase involves a human at a keyboard. Program sshpass strives just as mightily to defeat that, to allow a non-interactive test script. The actual passphrase can be passed to program sshpass by several methods, and the code sequences below show three of them. There can be spaces and special (non-alphanumeric) characters in these passphrases, but at the point where they are passed to command sshpass (or used in a command to create a password file), command-line interpretation of certain special characters may occur. For Windows protocol, file xxx.smb.conf can be an empty file.
For both http and https, program lynx is a text-oriented web browser.
For https testing, there is a simple CA-Certificate-Authority. File demoCA/toy-ca-cert.pem is the Certificate for that testing CA.
Basic SmartMirror, rsync protocol, can only be used with local userid defined on the ZipZerver, and cannot be used with DOMAINNAME identifiers. This code is included here for completeness. Note that the password file must not be group- or world-readable (no more than 0600 mode). See rsync_ssh below.
For each of the protocols that use ssh,
the UserKnownHostsFile should contain one or more of the public keys
of the ZipZerver.
If there is no such key,
ssh attempts interactive verification of the key,
which can be disruptive to a test script.
A minimal file that will avoid the interactive step
can be created by --
As with basic SmartMirror-rsync, the test file to be fetched, DMN_MARK_FILE, is not displayed directly but goes into a directory LOCRSYNC on the testing system.
For FTP-File-Transfer-Protocol
(unlike the other protocols),
the TESTU userid value must contain two "\\"-slant chars,
such as DOMAINNAME\\userid,
and (as described above)
must be set for a typical Linux/Unix command shell with --
FTP_ssh (unlike basic FTP) has TESTU userid with DOMAINNAME\userid,
with a single "\"-slant char,
and must be set for a typical Linux/Unix command shell with --
As with rsync_ssh,
a minimal UserKnownHostsFile
(that avoids a possible interactive key verify step)
can be created with --
This use of program sshpass uses a password file, shown here as RSYNCPASSF. Like with rsync, this file must not be group- or world-readable (so the file mode can be no more than 0600).
For ssh used for a single command or for a terminal login session
(and similar to rsync_ssh and ftp_ssh),
the UserKnownHostsFile should be set
(that avoids a possible interactive key verify step)
and can be created with --
Running a (limited) Active-Directory Domain-ControllerUpdated 2023-July-08, to remove this feature.Version 1.2.10 is the most recent version of this firmware which includes this feature, which involves a great mass of programs and libraries that are otherwise unneeded. Since the primary use for this mode was testing other ZipZerver firmware in an environment that did not happen to have any MSFT AD_DC, the extra build complexity and bulk seems like a poor choice. The (now obsolete) text below remains (for now), for historical reasons. The ZipZerver can run as an A-D Active-Directory D-C Domain-Controller in a limited way, primarily by providing DOMAIN\userids and password checking to users of that domain. This is useful for testing A-D membership during ZipZerver development, and may or may not be useful in any other context. A-D Policies, home directories, profiles, groups, file-level permissions, back-up-controllers, etc, are not provided. The device used for /flash files must not be too small. The files created by A-D quickly approach 60MB, with a rate of growth not currently known. With at least 2 versions of the firmware at 45MB, the previously-workable small limit of 128MB is insufficient. The /flash device is expected to be small compared to the size of any Raid-Group, and is expected to be 16GB or less. For A-D D-C use, a /flash device from 2GB to 16GB should be okay. It is a good idea to run System -> Date/Time Automatic Time Set/Synchronization in Continuous monitor and adjust mode. The samba docs mention the possibility of Kerberos tickets, used as part of the security system, being considered invalid due to out-of-range timestamp. If a local time-server is not available, then either [0.1.2.]pool.ntp.org or time[1234].google.com can be used. The transition to A-D D-C that is (somewhat) tested is from Stand-alone server. It may be desirable (but not required) to first invoke the samba-state off with data discard. Among the files thus purged is the one holding the samba version of the passwords for the local userids. Any such previously-created ZipZerver local userids will still be listed on the admin web pages, but will not be usable from samba until re-created with new passwords. Thus it will work best if the DOMAIN\userids are created after the transition to A-D D-C. The Windows Workgroup/Domain Name is a netbios type name. It works if it is the first component of the Realm name, and perhaps there are circumstances in which it is not just the first component of the Realm. The Active Directory Realm may be an actual DNS value, such as "polycosmic.mynetgear.com", which the global DNS system will resolve to an actual IPv4 and/or IPv6 address. It might also just look like a DNS value, such as "polycosmic.local". As an A-D D-C, the ZipZerver will be running a DNS name server, and it will resolve the Realm/Domain name (legal or otherwise) to itself, along with resolving the names of domain members and of A-D services. For A-D services on the D-C to work, the Network -> TCP/IP web page must have a DNS Name Servers entry for itself, to make use of the Name Server that A-D will run. (It might work to use the 127.0.0.1 loopback address, but it definitely works to use the LAN-visible IPv4 or IPv6 address.) It may work to set this entry after A-D D-C service has started, but it definitely works to set the DNS Name Servers entry first and use the Save Config/Next Values submit button (rather than the Apply Now button). On the Network -> Windows page, when Apply Now is (eventually) done for A-D D-C, the DNS Name Servers list is updated as part of the process. Speaking of DNS, each of the machines that will become members of the domain must also use, directly or indirectly, the name server run on A-D D-C. For other ZipZerver that will become members of that domain, the Network -> TCP/IP admin web page DNS Name Servers entry can be set pointing to the A-D D-C. For other machines that are set using DHCP, it might be possible to set the DHCP message to identify the A-D D-C as the preferred name server. It may be necessary to adjust the /etc/resolv.conf or equivalent on each such machine. If re-config for DNS Name Servers for all possible member machines is too much of a hassle but running a DOMAIN\userid login service still seems like a good idea for a small network, then perhaps Run a (limited) NT-type domain would be a workable choice, since netbios names can be resolved by broadcast messages and do not need DNS Name Servers to be carefully configured. If Domain User for Join is set to "Administrator" and that userid does not exist at the time when A-D D-C is first run, it will be created for samba use with the Domain User for Join Password value, which does seem to violate one of the few near-universal interface standards, that when creating a password it should be entered twice, so that there is some reason to believe that the person creating it knows what it is. This userid and password will be used at machines that are attempting to join the domain. It is unclear if it must be "Administrator", "administrator", or can be something else. If you have not (and do not) create a corresponding ZipZerver local userid, then that userid would not be usable with any of the other protocols (of course), but still would be usable through samba, and would not appear in the admin web pages list of local userids. Given the number of odd things that can occur in the A-D D-C mode, a Syslog Message Level of 2 has proven to be useful. The step to run as D-C for the first time can take a minute or more, as various database files are created and initialized. Setting SMB/CIFS protocol off (without choosing discard) and then restart is quicker, but still can take several seconds. After the node is running as A-D D-C, any userid created on that ZipZerver will be directly usable as DOMAIN\userid from any machine that is a member of that domain, from almost all of the protocols. This facility is primarily expected to be used for testing ZipZerver use of DOMAIN\userids, and there is expected to be a distinct shortage of admin-friendly tools for managing this domain. There might be conditions in which one of the command-line tools is useful. Program samba-tool has many options that are possibly useful. Either from the console (after Alt-F2) or by ssh, login as admin for a command-line prompt.
The program is located in directory --
There are some options which do not require root privilege
and can be run directly by admin.
If there turn out to be useful or necessary things
that do require root,
then the privilege tables will be adjusted to be able to use --
Discussion of the care and feeding of an Active-Directory domain by using samba-tool is beyond (way beyond) the scope of these notes.
Server for Macintosh AFP Apple File ProtocolUpdated 2012-Sept-05 The Help for the admin web page with the link Network -> Information has a general discussion about Enable for multiple protocols. For Macintosh AFP Apple File Protocol, previous versions of the firmware had a reasonably full implementation. However, with the update of a number of ZipZerver elements, the AFP server has been dropped. Note that recent Macintosh versions can run Windows CIFS/SMB protocol as a client to a file-server, so that AFP use does not seem to be critical.
Server for Unix NFS ProtocolUpdated 2024-January-24The Help for the admin web page with the link Network -> Information has a general discussion about Enable for multiple protocols. For NFS, there are several interacting system tasks that would be eliminated by disable of this protocol. For the other network protocols, a client is a user (or at least has a User Account of some kind), with the access coming from some system which has a mostly-irrelevant IP-Internet-Protocol address. The clients of the NFS server, in contrast, are systems, identified by name or perhaps only by an IP address. Each of those systems could have one or more users identified by a User Account, but for an NFS server, the client is the system. Version 1.2.15 has the ability to restrict which NFS clients can NFS-mount each share. The links at Storage -> Shares -> Share_Name or Permissions for each share, which allows selection of Read-Write or Read-Only, Public or Non-Public, and the list of Userid accounts which can access the share, now has a field NFS Client Addr/Names to set individual system names, IPv4 addresses, IPv6 addresses, and/or subnetworks by IPv4/masksize or IPv6/masksize. The section Available for NFS mount will show the names that can be used by a client system for an NFS-mount for each share that has Unix-NFS protocol enabled. This will also show which named systems, IPv4 or IPv6 addresses, and IPv4/masksize or IPv6/masksize subnets are allowed for such an NFS-mount.
In earlier versions of this firmware,
the mount-names were of the form
For at least one NFS client, any of these three forms could be used. For clients running only NFSv4 (rather than v2 or v3), the variation that begins "/hd/..." may not work.
In general, an NFS client system can find this list with the command --
These names would be used on the client system with a command of the form --
The equivalent effect can be achieved with an appropriate line within the client system's /etc/fstab file. This section will display "(file is empty)" if there are no shares with Unix-NFS protocol enabled. The syntax for the NFS Client Addr/Names field (on the Share-Name Permissions admin web page) allows system names to be used which can be resolved and even certain kinds of wild-card names (with "?" or "*" as part of the name to match any one character or any substring). This depends on the name-resolution mechanisms for the network environment, which will have the definitive word on what is valid and what is not. The section Most recent export operation with any error messages will show the result of the export operation, and will show any error messages about the Client Addr/Names field value. With or without any detected problems, the set of export names is displayed again in the section Most recent export status with details for RW, RO and other options, this time with a list of the non-default export options that are available for each permitted client system. Among those will be "rw" for read-write or "ro" for read-only, which should reflect the read-only attribute of the share. There are a number of export choices that could be adjusted for all NFS-enabled shares on a ZipZerver with the data field NFS Options (and with special syntax, on a per-share and per-client basis, within the Client Addr/Names field of the Share-Name Permissions admin web-page). This should not be attempted without expert advice. The section NFS Required lists the choices necessary for ZipZerver operation, for reference. Among those are "all_squash" and "root_squash" (which names are clear evidence that NFS was created by engineers with a problem to be solved, and not dreamed up by a marketing department), so that all accesses from these client-systems use the same userid. That userid is "public", so that files and directories have the same ownership regardless whether creation was done by NFS, Windows, SmartMirror, or FTP. (The command "ls -l" within a ZipZerver NFS-mount will list the owner and group name associated with userid=10 and group=10 on that client system, which is "uucp" and "wheel" on the development system. Do not be alarmed.) Among the export choice options that could plausibly be adjusted --
Server for FTP File Transfer ProtocolUpdated 2018-December-23The Help for the admin web page with the link Network -> Information has a general discussion about Enable for multiple protocols. For Server for FTP File Transfer Protocol, there are virtually no resources allocated for running the protocol until a client appears. The Server for Encrypted FTP over ssh is independent of non-encrypted FTP, and each may be enabled separately from the other. They differ in major ways, however. Like the other protocols, the non-encrypted FTP server runs as userid public (regardless of the login credentials), the resulting files are all owned by userid public (and thus shares can be changed from private to public without ownership issues), and the server-programs for each protocol enforce which users can access which shares. Encrypted FTP uses ssh-Secure-Shell for a login, and thus runs as the selected userid, and then access is allowed to all directories and files that Linux/Unix file permissions permit. This works well for a typical multi-user system with files owned by individuals and groups and file access permissions set for sharing or for restriction, as appropriate. On the ZipZerver, however, in which most of the userid are variations of public, this means Encrypted-FTP can access all Shares and all Files, regardless of whether public or private. An additional wrinkle -- an Encrypted-FTP session used by admin would create files owned by admin, rather than by userid public. This could lead to ownership issues -- e.g. those files could only be deleted by admin. There are a few controls -- Encrypted-FTP can be enabled as a read-only service, or as a read-write service, or left off. If on, the protocol can only be used by userid marked with FTP-ssh operation on the Security -> Share-User-Permission web page, or Encrypted FTP over ssh on the Security -> List Users web page with details for each individual userid.
On some client systems,
Encrypted-FTP would be accessed with a program named "sftp", for "secure-ftp".
For interactive use, it may be sufficient to use --
To avoid an interactive confirm step for the ZipZerver Public key,
you might be able to use the steps --
Repetitive test scripts might be able to use --
Domain Userids for FTP and Encrypted-FTP Protocol For FTP protocol, for a ZipZerver named NodeName joined to an A-D Active-Directory domain named DOMAINNAME, a domain userid can be used where a local zerveruser can be, in various forms. Note that for FTP, the required form of DOMAIN\\userid uses two "\\"-slant characters.
For Encrypted-FTP, the required form of DOMAINNAME\userid uses only one "\"-slant character.
The full discussion of the form of userids needed for each protocol for both A-D Active Directory and for older NT-4-type domains is given with the help text Domain userids, Local userids, and Protocols
Server for SmartMirror Rsync ProtocolUpdated 2018-December-23 The Help for the admin web page with the link Network -> Information has a general discussion about Enable for multiple protocols. For Server for SmartMirror Rsync Protocol, there are virtually no resources allocated for running the protocol until a client appears. A connection made to a server program on any network node uses a port number, a value from 1 to 65535, with particular network services usually found at a particular port number. Server program rsync, used for SmartMirror operations, generally is found on port 873. There are times when an alternate port can be used, such as when multiple ZipZerver must share a single external IP address behind a NAT-Network-Address-Translation firewall-router. The field SmartMirror Port Number can be used to set an alternate port number. After setting a non-standard port, SmartMirror client tasks created on another ZipZerver can either use the AltPort field, or can use an older syntax for the Remote Address field of ip.ad.re.ss:altport. If the current ZipZerver has an IP address 192.168.1.100 and will be using a SmartMirror port of 871, then the Remote Address field for the Backup -> Create or Edit a SmartMirror Task web page on the other ZipZerver could be 192.168.1.100:871. To use IPv6 addresses which are generally represented using hex-values and ":"-colon characters, the syntax would use square brackets -- [2606:2800:220:1:248:1893:25c8:1946]:871 or [example.com]:871
If program rsync is used directly from the command line of another system
(Linux, Unix or other),
the command to access a share named "sharename" using the
standard rsync port would use a remote address --
In client server mode, SmartMirror works really well for hundreds or for thousands of files. It works, but not as well for millions of files, mostly because of building an in-memory list of the names of the files to be transferred, in whole or in part. The load on the memory of a ZipZerver can be substantial. To prevent too many concurrent SmartMirror server tasks from running and allocating too much of the system's memory, the field Max rsync connections can be used to limit the number of simultaneous SmartMirror server connections. If the limit is hit, additional connection attempts are refused. The value for Time Out is used when the client or server end has created the list of candidate files to process, and the list must be resolved with the other end. When used in directories and subdirectories that have millions of files, the difference in time between the quicker end and the slower end can be substantial. This value sets how long is too long to wait for the other end. Note that SmartMirror and rsync clients can use ZipZerver userids, as created with the Security -> Create User admin web page. Currently, "Active Directory" DOMAINNAME\Userids usable by other network protocols cannot be used by SmartMirror and rsync. The protocol Server for Encrypted SmartMirror over ssh is related (obviously) but independent of (non-encrypted) Server for SmartMirror -- each can be enabled or dis-abled independent of the other. For a client, such as another ZipZerver, to use Encrypted-SmartMirror, there must also be --
In addition, one or more of the Public Keys must be delivered to any client system and properly verified to be used in that client system's ssh known_hosts_file. The verify step may involve comparing the SHA256 fingerprint values for Public keys on the Network -> Other web page. And maybe a certain amount of trial and error. Domain Userids for Encrypted SmartMirror Rsync Protocol For basic SmartMirror protocol, only zervuser, locally defined on the ZipZerver, can be used.
For Encrypted SmartMirror, for a ZipZerver named NodeName joined to an A-D Active-Directory domain named DOMAINNAME, a domain userid can be used where a local zerveruser can be, in various forms.
The full discussion of the form of userids needed for each protocol for both A-D Active Directory and for older NT-4-type domains is given with the help text Domain userids, Local userids, and Protocols
Other System Management ProtocolsUpdated 2022-January-25 for SVN-Subversion The Help for the admin web page with the link Network -> Information has a general discussion about Enable for multiple protocols. The admin web page Other System Management Protocols provide controls for several network and system facilities. SSH-Secure-Shell can provide a secure, encrypted login to a command-line prompt, and could be used by ZipZerver Customer Support to help with unusual system diagnosis and maintenance problems. The primary expected use for SSH-Secure-Shell is to provide the login and encryption mechanism for Encrypted-SmartMirror, Encrypted-FTP, and for Encrypted SVN-Subversion, and those protocols cannot be used unless the Server for Secure Shell has been enabled. Because of the potentially disruptive effect of command-line ssh login, use of any of these four protocols requires specific permission to be granted to a userid, either on the Security -> Share-User-Permission web page in the Smirror/rsync-ssh, the FTP-ssh, the SVN-ssh, and/or the cmdline ssh columns, or the admin web page Security -> List Users with details for each userid by selecting the Encrypted SmartMirror/rsync over ssh, the Encrypted FTP over ssh, the Encrypted SVN over ssh, and/or the command-line ssh login checkboxes. A connection made to a server program on any network node uses a port number, a value from 1 to 65535, with particular network services usually found at a particular port number. The server program for Secure Shell is generally found on port 22. The field Ssh Port Number can be used to set an alternate port number. This may be useful when multiple ZipZerver must share a single external IP address behind a NAT-Network-Address-Translation firewall-router. An additional reason for use of an alternate port for Secure Shell is to reduce clutter in the Status -> Login History file. A system with a globally-accessible IPv4 address is likely to see repeated login attempts to Secure Shell on port 22 from IP addresses assigned to interesting places around the world. If Secure Shell is running on some unexpected port, perhaps 30401, the random login attempts will be greatly reduced. Security experts stress that this does not increase overall system security, since an attacker intent on you or your system can discover this alternate port, and the success or failure of an attack still depends on the strength of your passwords. In the absence of such a directed-attack, however, the clutter in the Login History log will be greatly reduced.
To make use of this alternate port number for command-line login,
a client Secure Shell program such as putty will need to be properly configured.
For a command-line program such as Unix/Linux ssh,
the syntax to use an alternate port, such as 30401,
to reach a ZipZerver at IPv4 address 192.168.1.100 would be --
The SSH Port Number also defines the Port Number that must be used for Encrypted SmartMirror and for Encrypted FTP. See the Help for Network -> SmartMirror and for Network -> FTP. To make use of SSH for any of these uses, the Server for Secure Shell must be set On, and keys unique to the system must be generated with the submit-button Save, Generate/Replace SSH Keys for Server, Apply Now on this web page. The count of Host SSH Keys starts at zero at the factory. Server for Secure Shell can not run until keys have been generated. After generation, there should be several keys, marked RSA, ECDSA, ED25519, and DSA. Earlier versions of this firmware generated a key marked as rsa1 which is now considered weak and is recommended to no longer use. The Save, Generate/Replace SSH Keys... button will discard that key, and any others, and generate a set of shiny, new Public/Private keys. In these keys, the public-part is represented by a long cryptographic hash value "fingerprint" labeled as "SHA256". A fully paranoid correspondent may ask you, using some out-of-band communication (e.g. over the phone, rather than e-mail), to read all or part of the fingerprint for one or more keys. This assures the correspondent that he is communicating directly with the correct system, and avoids possible "Man-In-The-Middle" eavesdropping on the secure connection. If there is ever any question about the security of the private part of these keys, that there might be an unauthorized copy of the private part taken from the ZerverModule, the SSH keys can be generated again, replacing the previous keys. The fingerprint should show a completely different value. The downside is that a previous correspondent who painstakingly verified the correct public key from the fingerprint of the previous value will get an alarming-looking message on the next connection-attempt warning that the public key is not the same as before, thus a Man-In-The-Middle attack may be in progress. For SSH-Secure-Shell command-line login, for a ZipZerver named NodeName joined to an A-D Active-Directory domain named DOMAINNAME, a domain userid can be used where a local zerveruser can be, in various forms.
The full discussion of the form of userids needed for each protocol for both A-D Active Directory and for older NT-4-type domains is given with the help text Domain userids, Local userids, and Protocols There are other system-management facilities controlled from this web-page. ZerverView is a PC program to find and manage Zervers of various kinds. The field Agent for ZerverView enables communication between that program and this ZipZerver. ZerverView can locate Zervers on the same LAN segment with the PC, keep track of Zervers from more remote IP addresses, can report an IP-address and perform a few useful operations, such as reboot. Unfortunately, ZerverView can only report on one IP-address of a ZipZerver. A replacement for ZerverView can run on a ZipZerver, and is found by following the link for ZView Manager. Unfortunately, the Agent for ZerverView must be off within a ZipZerver before ZViewManager can run and make use of the ZerverView port. SNMP stands for Simple Network Management Protocol, is quite a bit less simple than ZerverView, might be used in a network with hundreds or thousands of nodes, and can be used to collect performance data as well as availability data. To be used it must be Enabled, as shown in the field Agent for SNMP, and there must be a Community String which functions a little like a password. A frequent value in this context is "public" (but please do not tell anyone). To deal with hundreds or thousands of nodes, knowledge of the physical location is important. The fields Location to report for SNMP and Contact Person can have any sort of value, especially if it helps with the physical management problem. The Location field is also used, since it is available and handy, within e-mail alert messages and several Status admin web pages. Boot-up System Messages can be directed to the system console, which is where most people would expect to find them, but which can be distracting, especially if there are ominous-looking messages that say "Fatal" related to the well-being of some sub-subsystem which are not, in fact, Fatal or critical. These messages could also be sent to the first serial port, at 38400 baud, which was very handy during firmware development since one could scroll through messages from the previous shutdown, as well as boot-up messages. Almost all configuration changes within a ZipZerver take effect when an "Apply" submit-button is used. This selection, however, does not take effect until after the next firmware update and restart. SVN-Subversion is a source-code control system used in software development to keep track of a set of files, and different versions of each of those files, and has been used in the development of ZipZerver. (Another such source-code control system used with FOSS-Free/Open Source Software is git.) There can be only one share chosen for the Subversion Repository, and Subversion Share is used to select it. (This is different from the other protocols which can be enabled on one or on multiple shares.) There can be multiple projects created within that single subversion repository. For Subversion Enable, the selection Yes will allow subversion to be used if a suitable Subversion database already exists within the selected share. The selection Yes, and Create and Initialize a Subversion database if it does not exist is used to create that repository. It may be that a Share is created that will primarily be a Subversion repository (and also has SmartMirror enabled, of course, so that there can be a backup). Or it may be that an existing Share with multiple uses is suitable, but that the Subversion database should not be at the root of that share. The entry for Subversion Path allows for one or more directory levels between the Share point and the Subversion repository. There may be more to say at Subversion Use within ZipZerver
SNMP Scan -- Summary The controls for SNMP Scan provide a different sort of service. Instead of providing an SNMProtocol server-agent, as above (providing status values from the machine itself as requested by one or more other SNMP clients), SNMP Scan allows this ZipZerver to monitor the server-agents of any number of other machines. Once enabled, an SNMP Scan will be done every minute. Each IPv4 address to be scanned is defined by a probe file which provides the IPv4 address, the SNMP OID-Object-IDentifier values to be requested, and specifying a number of options and parameters to be used for the scan of that particular machine. Any number of probe files can be provided within the probe file directory. Each file must have a unique, useful name, so the IPv4 address is encoded as part of the probe file name, and there may be additional useful identifiers. Example -- probeSuperWidgit_192_168_33_99_zizzle1.txt The general form is "probe*_ip_add_re_ss.txt" or "probe*_ip_add_re_ss_*.txt", beginning with "probe" and with suffix ".txt", with the IPv4 address delimited with "_"-underscore characters. The "*"-wild-card characters can be empty, can be replaced with identifiers for the target machine type (such as "SuperWidgit"), to document the expected machine name (such as "zizzle1"), and cannot use either "_"-underscore or " "-space characters. The identifier left after dropping "probe" and ".txt" is the target-agent identifier, and it is used in SNMP Scan status messages to list which probes had success or timeout or other conditions, and is used to form the result file name to be written to the result directory. Example -- resultSuperWidgit_192_168_33_99_zizzle1.csv The "result" part of the file name is the default value for a parameter in the probe file.
SNMP Scan -- Configuration SNMP Scan is configured by selecting a Share Name which will contain both probe files and results. Within the share, an optional Directory for probe-files can be set, and an optional Directory for result-files can be set. One use for SNMP Scan could use multiple ZipZerver to monitor within multiple networks, with a SmartMirror task to pull probe files from a central configuration site and another, more frequent, SmartMirror task to push the result files in the result directory back to a central monitor site. Separate probe and result directories make that straightforward.
SNMP Scan -- Current/Recent Status The Current/recent status field shows a short summary or error message from the most recent SNMP Scan operation. This can be "Unknown" if SNMP Scan is not configured to start, or if the status is checked soon after such configuring or soon after reboot, before the operation has had a chance to run. The fields Timestamp of status and Time now will show if the status information was generated within the last minute, or is older than that. An SNMP Scan operation which found any target-agents which returned any of the requested OID values will list the target-agent identifiers in the field One or more OID values found. The target-agent identifer will be listed with Probe marked INACTIVE if there is an INACTIVE directive within the probe file. The target-agent will be listed with Cannot write result file if the attempt to open the result file fails. This can occur for permission problems, such as previously-written result files being changed to be read-only, or if something else already exists with the result file name, such as a directory. (During development, files created when userid admin manually ran an SNMP Scan could not be over-written when the SNMP Scan was run by userid public). If the probe file does not include any OID request values, the target-agent will be listed as Probe file bad format. Additional checks on the validity of the probe file contents may be added. Timeout can occur if a problem with the network leaves the machine unreachable, the machine is not running, or if the target machine is running but the SNMP agent is not enabled and working. Timeout may also occur if the COMMUNITY string has the wrong value. Timeout may also occur if any of the OID request values are unrecognized or unsupported by the target-agent (for -v 1 version-1 protocol use -- see further discussion with the COMBINESCAN directive with the sample probe file). The IPv4 address is taken from the probe-file file-name, and is expected to be the dotted-quad notation, but using "_"-underscore instead of "."-dot. If the value extracted is not an IPv4 address, such as from "probe_192_168_33_xx.txt", or is a proper IPv4 address which is network accessible but which does not have an SNMP agent running, an entry is made for Bad IPv4 address/Unknown host. The check for IPv4 validity is picky -- "probe_192_168_33_99.txt" is valid, but "probe_192_168_33_099.txt" was not. If -v 3 version-3 of the protocol is selected by probe file parameter SNMPPARAMS, the internal message is "No securityName specified", and the target-agent is listed with Security Context not given. It should be possible to use -v 3 version-3 if all the needed values are supplied with SNMPPARAMS. Any target-agent listed under Unrecognized result does not fit in any of the other categories. The result file has been written in the result directory, and can be examined for clues to the problem.
SNMP Scan -- Sample probe.txt file
# probeTurboWidgit_192_168_33_99_zizzle1.txt -- SNMP Scan parameter values for one target. # # Format of the file-name -- IPv4 address is encoded within the file name -- #### probe*_ip_adr_re_ss.txt #### probe*_ip_adr_re_ss_*.txt # # Format of this file -- lines with #-hash char in column 1 are comments and ignored. # Each keyword is scanned separately, so these fields can be in any order. ### An uncommented line of -- #INACTIVE ### would cause the scan to skip operation with this file. # SNMPPARAMS can specify snmpget program params to use for this file's IP. # These are added after a standard set of options, and thus would over-ride. # -v 1 or -v 2c or -v 3 -- specify snmp protocol version. # -OQ -- drop out result type string such as "STRING:" or "Counter32:" # -Oq -- drop out result type string and "="-equal mark. # -r retrycnt -- can override default value of -r 0 # -t timeout -- can override default value of 2 seconds. SNMPPARAMS=-v 1 -OQ COMMUNITY=mediumsecret # Any number of OID values can be specified to be fetched from the # target SNMP agent-server. # The actual full number is required. Names taken from MIBs, # standard or specialized, are not supported. # sysName.0 OID=1.3.6.1.2.1.1.5.0 # sysContact.0 OID=1.3.6.1.2.1.1.4.0 # sysUpTime.0 OID = 1.3.6.1.2.1.1.3.0 # snmpInPkts.0 OID=1.3.6.1.2.1.11.1.0 # snmpInBadCommunityName.0 OID = 1.3.6.1.2.1.11.4.0 # HOST-RESOURCES-MIB:: # hrSystemDate.0 OID=1.3.6.1.2.1.25.1.2.0 # hrSystemProcesses.0 OID=1.3.6.1.2.1.25.1.6.0 # hrMemorySize.0 OID = 1.3.6.1.2.1.25.2.2.0 ### An uncommented line of -- COMBINESCAN ### would force all OID values to be handled in one operation, ### rather than the default, one snmpget request for each OID. # RESULTTEXT can provide the string which replaces "probe" # when creating the result file name. The default value is "result" # An alternative string can be provided. e.g. -- RESULTTEXT=StatusOf RESULTTEXT=statusof # DATETIMEFMT used with date command to get an exact display format # The "+" is required to signal a format string, # but does not appear in the final result string. # %m/%d/%Y produces 07/24/2013 # %T produces 15:20:25 as this is composed. Other format values are available. # Other chars, including spaces, and trailing spaces on the line, are preserved. # Note that a ","-comma char could be included, to create comma-separated-values. ### e.g. -- DATETIMEFMT=+%m/%d/%Y %T, DATETIMEFMT=+ %m/%d/%Y %T # VALUEQ can be used to modify the "Value=" string that appears before # the actual result value. # To create comma-separated-values, e.g. -- VALUEQ=, VALUEQ=Value= Within a probe.txt file, an uncommented line of INACTIVE will cause the file to be skipped, and an SNMP scan is not done for that file. This may be useful for probe-files that have detailed content which is not quite right, or for which the target-agent is known to be unavailable. An alternative method is to rename the file to not match the "probe*.txt" pattern, such as "probeTurboWidgit_192_168_33_99_zizzle1.xx". An advantage is that the SNMP Status display will show those target-agent identifiers that are INACTIVE. SNMPPARAMS is used to set snmpget parameters for the target-agent for the probe-file. The SNMProtocol version might be different for different target-agents, and can be set with "-v 1", "-v 2c", or "-v 3". The default output from an OID request includes a data-type string before the value, such as "STRING:" or "Counter32:". Adding option "-OQ" suppresses the data-type string. This is mentioned here as a possible probe.txt option since it may be desirable to see the data-type for some target-agent, at least during development of a probe.txt file for a new kind of target. Option "-r retrycount" can set a retry-count different from the SNMP Scan default of "-r 0" (that is, do not retry). Option "-t timeout" can set a timeout, in seconds, different from the default of "-t 2" (that is, 2 seconds). For "-v 3" version-3 protocol, additional parameters may be needed for security and authentication. Within a probe.txt file, the COMMUNITY string is used as the not-exactly-a-password value for the target-agent. The OID-Object-IDentifiers specify the actual values being requested from a target agent. With the current version of SNMP Scan, the values must be given in full numeric form, beginning with "1.3.6....", rather than being able to use text names taken from standard or specialized MIB-Management-Information-Block, such as "sysName.0" or "HOST-RESOURCES-MIB::hrMemorySize.0". The OID will be requested in the order of appearance within the probe.txt file. The default method is to use a separate snmpget operation for each OID. If the target-agent does not exist, is unreachable, or expects a different COMMUNITY string, there will be a timeout on the first OID request, and the target-agent will be abandoned for the current scan, having returned no OID values. There can also be a timeout no-response if one of the requested OID is unknown to the target-agent and "-v 1" version-1 protocol is being used. In this case, there can be partial success, with values returned for all the OID requested before the unknown one. An uncommented line containing COMBINESCAN directive will combine OID requestes, perhaps into a single request operation. If all of the OID requested are known to the target-agent, the values are returned more quickly and efficiently and with less clutter in the result file. For "-v 1" version-1 protocol, however, if any of the OID requested are unknown to the target-agent, then there is a timeout no-response, and no values are returned (which is why COMBINESCAN is not the default mode). If "-v 2c" version-2c protocol can be used with COMBINESCAN, then any OID requests which are unknown to the target-agent do produce a response, a STRING containing something like "No such Object available on this agent at this OID". The operation will be quick and efficient. RESULTTEXT can provide the string which replaces "probe" when creating the result file name. The default value is "result", creating a file such as "resultTurboWidgit_192_168_33_99_zizzle1.csv". Using a line with "RESULTTEXT=StatusOf" would create file "StatusOfTurboWidgit_192_168_33_99_zizzle1.csv". Within the result directory, each line of each result-file with an OID result begins with a date-time value, used to distinguish recent values from stale. The DATETIMEFMT directive can be used to adjust the size, order, and delimiter for the date-time, e.g. "DATETIMEFMT=+ %m/%d/%Y %T ". The "+"-plus is required to signal a format string, but does not appear in the final result string. "%m/%d/%Y" produces "07/24/2013", "%T" produces "15:20:25" as this is composed. Other "%"-format values are available, described with the "date" command for Linux. Other characters, including spaces, and trailing spaces on the line, are preserved for result-file analysis by character-count. Note that a ","-comma char could be included, to create comma-separated-values, e.g. -- "DATETIMEFMT=+%m/%d/%Y %T,". Within each result-file line with an OID value will be "OID=1.3.6...0" to identify the value returned. By default, this is followed by "Value=", and then the returned value. VALUEQ can be used to modify the "Value=" string. To create comma-separated-values, use a directive line "VALUEQ=,".
Subversion Use within ZipZerverCreated 2022-January-25SVN-Subversion is a source-code control system used during software development to keep track of sets of source files, and versions of each of those files. It is large, complicated, and supports many more choices than are used within ZipZerver. A possible starting place for more information is at Apache Subversion within the apache.org web site. (An alternative frequently used with FOSS-Free/Open Source Software is git. Subversion has been used with ZipZerver development primarily due to accidents of history, rather than a careful analysis and choice.) Configuration and Enable for SVN-Subversion on the ZipZerver is described within the Help text for admin web page Network -> Other System Management Protocols Basic Concepts. The SVN-Subversion repository can hold files for one or more projects, and each such project can include (main) trunk and (various) tag versions of the set of files. Development within a project is done by "svn co" check-out of a set of the files of the project, generally from the most-current set. After a set of changes within the files of this "working directory," a set of files can be "svn commit" back to the respository. Another user (or that same user with a different working directory) can bring those committed changes into another working directory with the "svn update" command. If that other working directory has incompatible changes to one or more of those files, the conflict is flagged for the second user to resolve. The default case is that files are not "locked" within the repository. When all useful changes from within a working directory have been "svn commit" (and any remaining changes should be abandoned), the working directory can be deleted. Full subversion has a number of choices for communication between a repository and a working directory. The one used in ZipZerver is done on top of ssh-Secure-Shell, primarily because userid and password checking are handled in a way that works with the rest of the system. The ssh code is very picky about those things, with the result that operations that need to communicate with the respository require a password to be entered. (This may be annoying to experienced SVN users, since a full installation has the option of consulting a clear text password file, rather than insisting on password entry.) Disk space is not a critical resource, and the working directory has a copy of each file as it exists in the repository (in a "hidden" directory named "/.svn/"). This means that a number of operations do not need to communicate with the repository. In particular, "svn diff" to show changes between repository and working directory do not need to consult the repository. Also, "svn revert" to abandon local changes within a file, to return it to the repository state. Each "svn commit" operation is noted with a "revision number." There are some operations that deal with multiple commits, such as display of each commit message associated with a given file. Examples In these examples, ipaddr is the IPv4 or IPv6 address of the ZipZerver, or a nodename, if there is a suitable local DNS service; the subversion repository has been created; SSH-Secure-Shell protocol has been enabled and keys created; zerveruserid is a userid on the ZipZerver (or an ACTIVE_DIRECTORY_DOMAIN\userid) which has SVN-ssh permission.
Find and Manage Other ZipZerver on the networkUpdated 2016-July-26The ZView web page is similar to PC program ZerverView, which uses a simple protocol (perhaps too simple) to find and report the IPv4 address for any ZipZerver for which the ZerverView protocol is enabled (on the Network -> Other admin web page). If the ZerverView agent is not running on a ZipZerver, then ZerverView and this ZView web-page will not detect it. Because a particular UDP port number is needed (1099), to run the ZView web-page as a manager on a given ZipZerver, then the ZerverView agent must be disabled. That means that the given ZipZerver would not be detectable to ZerverView or another ZipZerver while using the ZView web-page, until the ZerverView agent is re-enabled. If the ZView web page displays Problem -- Could not establish use of socket for ZView scan, then the ZerverView agent is very likely to be enabled. Disable it in order to run the ZView Manager. The checkboxes (and data field) allow the ZView scan to be specialized in various ways. The item LAN -- Scan for Zervers anywhere on the LAN is the default operation. If scan throughout the LAN segment is not appropriate, or there are Zervers to be monitored that are not on the LAN segment, an IPv4 or IPv6 address can be entered to Add Enumerated Address, and an Enumerated Addresses checkbox will be presented when there are one or more such addresses. The checkbox ExtCommands -- Report on ethN beyond eth0 will use an extended command set to show IPv4 addresses for all network interfaces (not just eth0), and will show IPv6 addresses, but that will be effective only for ZipZerver systems that are running release 1.1.13 or later. In ZipZerver systems running older firmware, the Current Details event log will show the harmless message "discover: invalid request was broadcast -- 100". Note that the IPv4 extended network information, for ethN beyond eth0, can be obtained for one node at a time (including for older ZerverView agents) using the single-host command details. Use of extended commands is appropriate when there are multi-network ZipZerver systems that are running very recent firmware, to get a display of the IPv4 and IPv6 addresses used by all network interfaces from all ZipZerver systems. The checkbox WrongFormat can be used to identify ZipZerver systems running firmware release 1.1.09 through 1.1.12, which were created incorrectly and do not respond properly to ZerverView commands. Very little can be displayed from these systems -- the eth0 MACid identifier and an IP address (IPv4 may be shown as part of an IPv6 address -- "::ffff:192.168.1.99" for 192.168.1.99). On those systems, the Current Details event log will show messages "discover: Not enough data received 84 wanted 96" and "discover: invalid request was broadcast -- 0". On properly created system, the message "discover: invalid request was broadcast -- 7546112" will appear in the Current Details event log. These are harmless. The checkbox Unroutable -- Use Broadcast/Reply for LAN Zervers with unroutable static addresses can be used to locate ZipZerver systems that were configured with static addresses for some other network, for which routing does not work when connected to the current LAN. Using commands broadcast to the ZerverView UDP port for all nodes on the LAN, with the flag set in the command to use broadcast-reply, basic information about the node can be discovered, and the single-host command eth0_dhcp can be used to reset the address and routing for eth0. The associated checkbox I have read the Help page... is used to indicate that use of the Broadcast/Reply feature has odd consequences for other ZipZerver systems on the LAN, sending ZerverView agents into a loop which can consume CPU time and network interface resources, and could fill the event log files with useless clutter. See the full rant below in the section Recovery/Reset for all Zervers after use of Broadcast/Reply. The Auto-Refresh -- 60 seconds checkbox sets a mode in which the ZView Manager web-page is updated every minute. This may be useful to wait for the reboot of a ZipZerver that takes a long time to re-appear. In this mode, WrongFormat and Broadcast/Reply are not allowed. The display of the ZipZerver systems found will be alphabetical by NodeName, and will show the NodeType, FirmwareVersion, and details about the eth0 network configuration of that node. Details for other ethN interfaces can be found with a display selecting ExtCommands (for newer FirmwareVersions, explained above) or with the single-host command details (described below). The recvfrom field is the IPv6 address from which the node information was received. This will be blank for the ZipZerver running the ZView Manager, and might differ from the eth0 IPv4 address if the node info was received from an interface beyond eth0, or if the node has made a transition from a static address to a DHCP address and can (for a time) respond to both. The ethid-mac0 value is the 48-bit MAC address from eth0, and is used to uniquely identify a node, both for commands and responses. The status field can display "Self" for the node running ZView Manager, "Unroutable" if Broadcast/Reply was needed to reach the node, "NoExtCmds" if an extended command was tried and failed, "Ok" if all seems well with the node, and "WrongFmt" for nodes that respond when WrongFormat tests are part of the scan. For WrongFmt, only very limited data is available -- the recvfrom address and the ethid-mac0 value. A limited number of operations are available that can be applied to individual ZipZerver in this list (but not to the Self node nor to any WrongFmt nodes). These commands require a userid with admin privileges defined on that node, and the associated password. The details single-host command will retreive IPv4 info about all configured network interfaces (rather than just eth0), and uses a previously defined command set, and so should work with older ZerverView agents (unlike the extended commands which are only implemented in newer agents). The eth0_dhcp single-host command will set eth0 to use DHCP to obtain an IPv4 address, and is expected to be useful for ZipZerver systems that were given static addresses that are unroutable on the current network. This use requires the Broadcast/Reply feature, and thus would lead to a need to check and perhaps reset the ZerverView agents on all ZipZerver systems on the LAN (as described below). The reboot and shutdown single-host commands are available when needed.
Recovery/Reset for all Zervers after use of Broadcast/Reply
Summary -- After a sequence of operations in which the Unroutable ... Broadcast/Reply feature has been used, check all other ZipZerver systems on the LAN, especially those running older firmware, to briefly disable and then re-enable ZerverView protocol, to stop a loop which wastes CPU time and network interface bandwidth. If the ExtCommands were used with this feature, do this within a few minutes to avoid completely overwriting several log files with userid-password error messages.
The Unroutable ... Broadcast/Reply feature is intended for a situation in which a ZipZerver has an IPv4 static address assigned for some other network but which is not routable on the current network. For example, a ZipZerver might have the static IPv4 address 10.10.10.99 due to prior use on a 10.10.0.0/255.255.0.0 network, perhaps with a gateway of 10.10.1.1. If that ZipZerver is connected to a 192.168.1.0/255.255.255.0 network, smooth communication does not occur because the routing tables on that node will not work. Such a node can be located by using broadcast packets for each command, to be delivered to all hosts connected to the LAN segment, setting the option that the Reply is also to be broadcast. This works, and can be very useful when alien ZipZerver systems must be recovered and adapted. The odd consequences occur because of the unsophisticated nature of the ZerverView protocol, specifically, because there is no fully reliable way to distinguish a command from a reply. When a ZipZerver broadcasts a reply, including its eth0-mac unique identifier, that reply is delivered to all nodes on the LAN segment, including the ZipZerver which just sent it, where it will look like a command intended for that specific ZipZerver, since it contains the eth0-mac unique identifier. Any other ZerverView agents that receive this reply note that the eth0-mac value is not for them, and ignore it. Only the ZipZerver running the ZView manager is expecting a command-reply, and correctly inteprets the message. Current firmware versions contain heuristics to recognize these bogus commands, and ignore them. (Definition of "heuristic" -- Mostly it works. In some circumstances, it doesn't.) Older versions do not contain these adaptations, and interpret the reply as a command. If that reply contained an error status, it will be mis-interpreted as the flag for broadcast-reply, and this sequence will repeat indefinitely. If there was not an error status, the mis-interpreted command will generate another reply to the address from which it was received, which is itself, and this sequence will repeat indefinitely. The affected systems do not crash, and should continue to respond normally to all other protocols, however the extra CPU and network processing load will lead to slugish response at some performance level. Most such Broadcast/Reply commands do not create any entries in the Current Details event log, but an examination in Status -> System Status -> System Status Internal Details will show several hundred PID process-id values per second -- check the left-column PID number for program systemdetails, then update the display after a few seconds. If the PID number changes by a dozen or so, all is well. If it changes by several hundred for each second, then the ZerverView agent is very likely in a loop. Also, values of 1216 (or larger) in the Recv-Q for the ZerverView UDP port (0.0.0.0:1099) would indicate a problem. If the extended commands are used with Broadcast/Reply, older agents that do not recognize those commands will attempt a userid-password check on some empty fields, fail, and add entries to Current Details, Login History, and Major Events Log. As a password-fail, there is a pause of 2 seconds (to slow down automated password-guess attempts), which is a much smaller load at 30/minute (compared with 300/second), however all useful historical information within the event logs can be overwritten within a few minutes. Thus -- after using Broadcast/Reply to find a ZipZerver with an unroutable static address (either from the ZView web page or from the ZerverView program itself), each older system should have its ZerverView agent disabled and then re-enabled. This has the effect of discarding the never-ending reply message. It may also be worthwhile to check ZipZerver systems with current firmware to see if the Current Details event log is being rapidly filled or the CPU has an unexpected load. Very old FileZervers did not provide a way to check if there were some runaway process, other than that the internal temperature reading might be unexpectedly high. Then again, there was no way to disable and then re-enable the ZerverView port without doing a system reboot.
Devices and PartitionsUpdated 2014-Oct-02The Devices admin web page shows what disk-devices are known, and what partitions, if any, are defined on each device. Partitions which are not in use can be selected for deletion, which is needed before that disk space can be used in creating a new Raid-Group. Information about each device and partition is displayed -- the Type and Usage may appear as "linux_raid_member" for partitions that are part of a Raid-Group, or "ext3" for the partition holding the ZerverModule flash file system. Version number for "linux_raid_member" may be "0.90" for Raid-Groups created with older versions of ZipZerver firmware, or "1.2" for more recently-created Raid-Groups. (Versions "1.0" and "1.1" might also be found in some circumstances.) Label information for recently created "linux_raid_member" partitions may appear as NetworkName:RaidGroupName, with NetworkName as the name of the node when and where the Raid-Group was created. Newly-acquired disk devices can show a much wider range of Type or Usage, including fat32 or ntfs filesystems. In at least one instance, a flash drive had a factory-created ntfs file-system associated with the device, rather than with a partition of the device. As many as 15 partitions can be displayed. Previous versions of this web page were limited to disks that were 2TB or less, using the (generally standard) MSDOS partition table. (The size limitation was due to that partition table's use of 32-bit offset values identifying 512-byte sectors.) Now disks of any size can now be handled. Also, previous versions of this web page used only the 8-bit type field associated with the MSDOS partition to provide information about the partition contents, rather than the Type, Usage, Version, and Label information. To use a disk-drive when creating a Raid-Group, the partition table must show that the disk is fully available. If there are partition definitions left over from earlier use in a ZipZerver, or earlier use in another system, or in-place from the disk factory expecting use in another system, those need to be deleted. For each disk, there is an entry for as many as fifteen partitions showing the size and type. If the disk does not appear to be in use, there will be a radio-button to select each partition to be deleted, or to delete all of the partitions on the disk. Select Delete Partition to delete one or select Delete All Partitions to delete all partitions (from 1 to 15) from that disk, and hit the Delete Partition Now submit-button. The partition table on each disk is very small, but the implications of a change are very large, for a number of kernel data structures. The update can take several seconds, and is done in a way that all of the associated kernel structures are updated properly. Perhaps because there is in-memory representation of this critical data, the Devices web page does not do a good job of showing failed devices, showing the state of the partition table as it existed the last time the device was dealt with, even for devices that are now missing or dead. For a better indication of the health of disk drives, check the List RAID Groups web page.
RAID GroupsUpdated 2023-Sept-06 (Mismatch count discussion) The RAID Groups web page shows the currently-known Raid-Groups, the status of each, and provides controls for manipulating a Raid-Group, a File-System within a Raid-Group, and the disk-drives of a Raid-Group. RAID is an acronym standing for "Redundant Array of Inexpensive Disks" or possibly "... Independent Disks". There should be a reference here to an authoritative source which defines the acronym, defines the variations, and describes the advantages and trade-offs with each. More information about different types of Raid-Groups is included with the help associated with Create RAID Group web page. If such an operation has been initiated, the current status is shown as Create Raid-Group and File-System. The web page List RAID Groups is intended to describe everything related to each Raid-Group and the associated File-System. The most-likely configuration for a ZipZerver will be one Raid-Group, using all disks, perhaps with one marked as a "spare". However, everything works fine with multiple Raid-Groups on a system, with details of each Raid-Group listed in different columns. The Raid-Group Name is the value chosen during the Raid-Group Create operation. The Raid-Group id is an internal identifier, md0, md1, ..., in which "md" stands for "multi-disk", and is provided here because some kernel error messages or status displays might use the mdN identifier, rather than the Name. The field Overall Status is actually a summary of other status fields from this page, and is intended to be a quick-way to see if the status is "OK" or something else. Values --
Field Array_state considers only the state of the software-raid-driver (nothing to do with the File System). The value "clean" is good. The field Type shows checksum/raid5, raid6, mirror/raid1, striped/raid0, jbod/linear, or several variations for raid10 with either two or three copies of each File System block. The field Drives shows the current number and the desired number for the redundant Raid-Groups. A value such as "8 of 8" is good. A value like "7 of 8" is not so good. If a Raid-Resync is in progress, either because of initial construction of a Raid-Group, after a spare drive has been added to a Raid-Group that had been in Degraded mode, or a manually initiated Sync Check or Sync Check and Repair operation, then there will be rows for Completed, showing the percentage complete, and Remaining, which shows an estimate of the minutes remaining before the full scan is complete. These numbers can be very helpful, but should be interpreted with caution. For example, on initial Raid-Group and File-System creation, the progress on the Raid-Resync is deferred in favor of creation of the File-System structures. The time-remaining estimate can be days or weeks, based on slow progress up to that point. Once the File-System structures are complete and the File-System is Available on-line, the Raid-Resync will begin to make much better progress, and within 3 - 4 minutes may show progress at a rate of 100 GB/hour (writing speed), with completion time of 300 minutes for a RAID-5 or RAID-6 made from 500 GB disks. If a heavy File-System load is put on that system during the scan, then progress will be slowed and the estimated end-time may (again) become higher, perhaps much higher. There may be a field Mismatch Count if there has been a manually initiated Sync Check or Sync Check and Repair operation since the Raid-Group was assembled (using the Begin Sync Check Scan of all Drives or Begin Sync Check and Repair Scan operations) and there is a non-zero count for the number of Check operations with a Mismatch -- either an incorrect parity block for RAID-5 or RAID-6 or mismatching data blocks for RAID-1 or RAID-10. The Check operation is done on a number of blocks at a time, perhaps 128, and if there is any mismatch during the operation, the Mismatch Count may be increased by 128. If there is no field displayed for Mismatch Count, then the value is zero; either no Sync Check operation has been done, or the most recent Sync Check found zero problems. If a full synchronization has been done, at Raid-Group creation or later (with Sync Repair), then the Mismatch Count for a full scan would be zero or small, and related to actual problems developing with a disk. (The Linux md-multi-disk documentation says that there is also a very small chance that an inconsistency could occur on RAID-1 or RAID-10 by a real-time race in updating a data block in memory and writing the multiple copies.) If the Mismatch Count is a non-trivial fraction of the number of blocks in the Raid-Group, the most likely reason is that an Initialization Synchronization was not done when the Raid-Group was created. This is legal and very workable for mirror-RAID-1 and for the various types of RAID-10 in which no file (and the File-System Blocks which hold it and the file's metadata) can be read unless it has been written first. Thus with or without a full Initialization Synchronization, everything that can be read by a proper File-System operation will have full, proper redundancy, and can survive a disk failure, degraded operation, and disk replacement and recovery at any time. (This absolutely does not work for chksum-RAID-5 and for RAID-6, since the checksum blocks which provide the redundancy for a stripe of blocks, one from each disk, seem to be updated using the change in each file block without reading and using all the blocks in the stripe. If there has been full initialization, this redundancy works well. Without full initialization, reconstruction using these checksum blocks after a disk failure produces junk. RAID-5 and RAID-6 must be fully initialized, either at creation or Sync Repair done before use.) The fields Blocks (1024) and MB (1000000) show the capacity of the Raid-Group space (in contrast to the size of the File-System which is held within that space). The value "1000000" is spelled-out so that it is unambiguously a decimal-million (in contrast to a power-of-two approximate million, as is sometimes more convenient to use within computer engineering). These numbers may be smaller than would be expected by adding up the values printed on the box that the disks arrived in --
File System Used MB is related to the File-System within the Raid-Group space, rather than the Raid-Group space itself. This would include file-content, directories, and space for the journal used for fast File-System recovery (32 MB), which is why a freshly created Raid-Group File-System shows 33MB used. Also note that since file-content is held in 4096-byte blocks, a system with a very large number of very small files might have much less in content than this number would indicate. The fields File System Free MB and File System Free, which shows percentage in part so that there is a number that can have highlight in yellow, show space available for file-content and directories. Note that the procedures for placing appended file data "near" the rest of the file cannot be very effective when the file-space is low. Even though 1% of a TByte space is still a lot of free space, there are performance reasons to not let it get that low. The fields File System Used MB and File System Free MB together will not match the field MB (1000000) --
File System Status shows any need for a File-System check operation. Value can be "clean" or "in use" for good status, or "not clean" for not-so-good. The value "clean with errors" can occur if File-System operation encounters a problem (which would be shown in the Major Events Log). A File-System Check operation should be done at an early opportunity to perform repairs on the File-System structures. A field File System Check will appear, with value "started", if such an operation is in progress, either manually initiated, or due to serious problems at system start-up. If the File System is mounted and available on-line, an entry appears for Available As, showing the name of the mount-point. The File System must be mounted in order for any of the share-points within it to be visible. For a File System named "raidblue", the mount-point is most likely to be "/hd/raidblue", using the File System name. There could be a variation if a system is booted and finds disks from two or more Raid-Groups which happen to have the same name (as can happen in an engineering lab with test scripts which create Raid-Groups using the same names over and over). In such a case, you could see Available As "/hd/raidblue" for a Raid-Group and "/hd/raidblue1" for another. If the File-System is not on-line, perhaps because it has been manually taken off-line, the field shows "(Not Available)". The File System Type field is "ext3" or "ext4" to show that a journal-File-System is in-use. The long-used "ext3" version is limited to 16TB file-systems. The limit for "ext4" is larger. The Disks and Partitions area is a list of all disks, providing an internal identifier (hda, hdb, hdc, ..., sda, sdb, sdc, ...), the I/O path to the disk (e.g. ide1.0, scsi4:0:3:0), the disk size in blocks (1024-bytes) and MB, and the text-description reported by the disk (e.g. Maxtor, SEAGATE) or by a controller (e.g. 3ware Logical Disk 3). The Disks and Partitions area is intended to show the relationship between disk-partitions and Raid-Groups for one Raid-Group, for multiple Raid-Groups which use only one partition on a disk, or even for multiple Raid-Groups in which a disk could have multiple partitions used in separate Raid-Groups (created by some method different from the ZipZerver interface, and then installed in a ZipZerver). In the column for each Raid-Group, any disk-partition which is part of the Raid-Group is shown in the row with the other disk information. There is an internal identifier for the partition (hda1, hdb1, ..., sda1, sdb1, ...), the partition size in blocks (1024-bytes) and MB. There is also a field [k] that shows the relationship of the partition to the Raid-Group. Values include --
Within Disks and Partitions beyond the columns for each Raid-Group is a column marked Select with radio-select buttons, one for each disk-drive, used with some of the Raid-Group operations for repair and replace, described below. There are also entries marked other which show for each disk the space, in blocks (1024-byte) and MB that is not in use by Raid-Group partitions. There is a lot of useful information in this display, and it can be lengthy, so the fields Raid-Group Name and Raid-Group id are repeated above the selectors for various Raid-Group and File-System operations, so that if there are multiple Raid-Groups, there is a better chance that the right one will be selected. Some operations cannot be done while a File-System is Available on-line and perhaps in use, and there may be other reasons why a File-System should be made un-available at some time. The File-System operation Take File-System Offline if not busy will succeed if there are no clients using the File-System or any files on it, and will fail with a message about "Busy" otherwise. The operation Take File-System Offline Now signals those client-tasks to relinquish use of the File-System (using a function named "kill"). Note that the "Now"-operation can still fail, with message "Busy", if there is a root-user operation using a file or with a working directory within that File-System. There may also be problems taking a File-System Offline if there has been NFS-Network-File-System access within the File-System. For these File-System operations, you select the desired operation for the desired Raid-Group, and then hit the submit-button marked Apply Raid-Group Change Now. With a File-System Offline (no longer Available), Run File-System Check is possible. A File-System can be a complex structure, with a very long lifetime for operation results, and sudden-shutdown, hardware-glitch, and software imperfection can leave odd problems. Use of a Journal, as part of the ext3 or ext4 File-System, eliminates much of the problem of a sudden-shutdown -- after the pending changes in the Journal are applied at system re-start, the File-System can be known to be in a clean state, just as if a proper shutdown had been done. Still, there are times when a series of seemingly in-explicable problems occur, and it would be prudent to take the File-System Offline and run a File-System Check, to find and fix any inconsistencies. The ZipZerver is configured to make File-Systems which do not call for automatic File-System check after some number of re-boot, or during a re-boot after some number of months, since multi-TB File-Systems can take a half-hour at best, many hours at worst, and this would be a really unpleasant consequence of a re-boot intended to clear some simple condition. Thus the capability to run a File-System check manually, at a time of your choosing. There will be a File System Check status line with a value of "Started" if one is in progress. There will be no such line or no value when the File System Check is done, and result log can be seen on the web page Status -> FileSys Op . To put the File-System back Online, so that it will be Available for clients, use the submit-button Assemble All Raid-Groups, Enable All File-Systems Now. There has been a request to add an operation to initiate a File-System Check and then automatically put the File-System Online if there are no problems. The operation to Stop the Raid-Group can be selected if the File-System is Offline, and after Apply Raid-Group Change Now is hit, has the effect that the Raid-Group is no longer assembled, and will disappear from the Raid-Group display. This is a step in using those disks to create a new Raid-Group. The next step would use the Devices web page to delete the partition definitions on each device. (After Stop the Raid-Group, the data are not gone, and the Raid-Group can be re-assembled and the File-System made Available Online by hitting the submit-button Assemble All Raid-Groups, Enable All File-Systems Now.) The operations manipulating individual drives within a Raid-Group can be done while the File-System is Available Online or is Offline. Mark Selected Drive as Failed can be done if a drive is suspected of having problems, or to test Raid-Group recovery mechanisms, by selecting the operation with its radio-button, selecting the target drive in the Select column, and hitting the submit-button Apply Raid-Group Change Now. The result will be a drive marked "FAIL" in various ways, with a tag [k] FAIL. Raid-Groups with redundancy should continue to operate. Raid-Groups with no redundancy, like raid0, will not. To Remove Drive from Raid-Group, the drive must be either a Spare or marked FAIL. The drive is chosen with the radio-button in the Select column, the operation is selected for the Raid-Group, and hit the submit-button Apply Raid-Group Change Now. Add Selected Drive as Spare is used to add an unused drive to a Raid-Group. If that Raid-Group had been in degraded mode, the arrival of a Spare will initiate a Raid-Resync to write the newly-arrived drive, so that full redundancy will be restored. The drive is chosen with the radio-button in the Select column, the operation is selected for the Raid-Group, and hit the submit-button Apply Raid-Group Change Now. Re-Add Drive to Raid-Group can be used in some circumstances for a drive which has been removed from a Raid-Group to re-add back into the same Raid-Group. This can sometimes be done without having to start a full re-sync of the drive. If there are Raid-Groups with redundancy (parity for RAID-5 and RAID-6, duplicate blocks for RAID-1 and RAID-10), three more operations are available. The Linux md-multi-disk documentation notes -- "As storage devices can develop bad blocks at any time, it is valuable to regularly read all blocks on all devices in an array [Raid-Group] so as to catch such bad blocks early." Sync Check, the Begin Sync Check Scan of all Drives operation will scan all blocks of all disks in the Raid-Group checking for consistency. If there is a read error, the normal read handling will find the proper data and re-write it, which may fix the data error. If there is no read error but the data is inconsistent, then Mismatch Count is incremented. Sync Repair, the Begin Sync Check and Repair Scan operation will also scan all blocks of all disks in the Raid-Group, repairing any read errors, and also fixing any data inconsistencies. If Initialization Synchronization was not done when a Raid-Group was created, the Sync Repair operation can be used at some later time to establish the same result. For large Raid-Groups, Sync Check or Sync Repair can take a long time, perhaps hours. During these operations, there will be rows for Completed percentage and Remaining minutes. There will be a non-zero count for Mismatch Count, if any are found. Operation Cancel Sync Check or Repair Scan will terminate any such scan in progress. Note that the result of using Cancel Sync during a Create Raid-Group Initialization Synchronization operation is undefined. The last submit-button, Assemble All Raid-Groups, Enable All File-Systems Now, is not associated with a single Raid-Group or single File-System or single drive, but with all drives and all Raid-Groups. Similar to the steps that occur at system start-up, all partitions on all drives (marked with the "Linux raid autodetect" partition type) that are not already in a Raid-Group are examined, any and all Raid-Groups are assembled and started, if possible. Then any and all Raid-Groups are checked for File-Systems and mounted Online, if possible.
Create a New RAID GroupUpdated 2023-Sept-09, rewrite full synchronization section The Name of Raid-Group field identifies the Raid-Group and the File-System within it, to distinguish between Raid-Groups on a system that have more than one, and sometimes to distinguish that a condition or error is related to a Raid-Group and File-System, as opposed to one of the share-names within a File-System. The Type of Raid-Group to Create selection is the critical choice which balances redundancy, capacity, and throughput. The choices as listed have a short summary of the advantages of each type.
A table below summarizes the different Raid-Group types, redundancy, File-System size and performance.
Checkboxes marked Use in Raid-Group appear with disks which have space available. Use these to select the drives to be used in the Raid-Group. Each disk is identified with several pieces of information, in which recognizable text might occur in the way that the disk identifies itself (e.g. "Maxtor", "Seagate"), or is identified by a controller (e.g. "Logical Disk 3"). Among the cryptic pieces will be internal identifiers (e.g. hda, hdb, hdc, ... or sda, sdb, sdc, ...), input/output path identifiers (e.g. "ide1.0", "scsi4:0:3:0"), disk-capacity as reported by various mechanisms expressed in "KiB" (1024-byte) blocks and in MB (1000000-byte) for the full disk, for any partitions already allocated, and for other available space. There can be disks listed without a select checkbox. Disks that are already in use will have no checkbox. Disks that are considered empty but that have had other use, either in a ZipZerver or in another system, or which have come from the factory preconfigured for another system, may have a partition table which allocates the disk space in various ways. To initialize these disks for use within a ZipZerver Raid-Group, use the Devices admin web page to delete the previously created partitions. A small-size device will be listed with an indication "/flash" for the ZerverModule that holds the firmware, configuration data, and long-term log files. It is not available for use within a Raid-Group. Currently, only one partition of a disk can be used in a Raid-Group. This means that a disk could have much space available, and there could be a Use in Raid-Group checkbox for a disk, but if the first partition of that disk has already been allocated, then that disk cannot be used in another Raid-Group. (Such a partition is shown as a line with an internal identifier such as hda1, hdb1, ... sda1, sdb1, ...) A feature to look forward to, perhaps. The field Size from each disk can be left empty to select the maximum usable space from each disk. This works directly for Striped RAID-0, even if the selected disks have different sizes. For the other Raid-Group types, the usable size is limited by the smallest selected disk. Leaving this field empty to select the maximum space for each disk works if those disks match in size. If the selected disks differ in size, then a size value should be set in this field. A near-maximum size with an attempt at optimal partition alignment can be achieved by setting a value in this field that is too large, and selecting MB, GB, MiB, or GiB. After the Check Selections step, a Warning message will indicate the maximum value for the selected unit size. There are reasons why one would want less than the maximum, primarily to create a Raid-Group and File-System of a particular size to match another File-System, for backup purposes. Creating a Raid-Group of a limited size is also useful for performance tests for different Raid-Group configurations. Setting the Size field may be required to avoid one error condition for those Raid-Group types that require (nearly) identical disk partition sizes. The error message is -- "mdadm: largest drive exceeds size by more than 1%". This condition causes failure of the Raid-Group Create very early within the process, within a minute or so. To recover, the newly-created partitions on each disk must be deleted using the Devices web page, the Create Raid-Group selections can be repeated, with an explicit value within the Size from each disk field. Code changes now provide the selected size value with the GB, MB, etc, size-unit value for the partition size, and this has the effect of allowing optimal partition alignment. With early versions of this firmware, the partition could start as early as sector 1, the partition size would be expressed in KiB, and there would be messages in Status -> FileSys Op log with text -- "Warning: The resulting partition is not properly aligned for best performance." If there is some circumstance in which the absolute maximum partition size is needed, with the lowest possible partition-start sector, the Size field can be expressed in KiB. This can still fail in the lower-level routines requiring a further small size adjustment for a disk in which the minimum partition-start is larger than 1 sector (48 sector minimum has been found in the lab). The size of the performance penalty for using a Size value in KiB (and for those Raid-Groups created by earlier code) may be indicated by two tests, in which a 400GB RAID-10-2X Raid-Group using disks with partitions that were poorly aligned took 128 seconds to initialize the file-system. Those same disks on that same system with partitions that were better aligned took 114 seconds. A non-trivial penalty, but not catastrophic. For RAID-5 and RAID-6 to provide fault tolerance and continue operating after a disk failure, initialization of all disks in the Raid-Group is required, with all checksum-parity blocks set. This is because the checksums used to compute the contents of a block for a failed disk only work if the checksums for all stripes for all blocks are in a known state, fully initialized, and then are maintained as blocks for files are written. For Mirror-RAID-1 and RAID-10, recovery from a failed disk does not require full initialization, but is still a good idea. There are two ways to create the proper condition. The checkbox Initialize with full synchronization can be set during Raid-Group and file-system creation. This is simplest, but has the disadvantage that synchronization has begun while the file-system structures are still being written, slowing both. The Raid-Group Remaining Time shown for this operation can balloon to days or weeks, because of the slow initial progress. The other way to achive the proper condition is to create the Raid-Group without full synchronization, and when the file-system is ready, use the Raid-Group operation Begin Sync Check and Repair Scan. The file-system will be ready a bit quicker, but the disadvantage is that you have to remember to do this. For RAID-5 and RAID-6, disk failure recovery will not work until it is. Either method can add hours (depending on the size of the disks) to the time before the file-system is fully ready for use. For Mirror-RAID-1 and RAID-10, disk-fail-recovery works without a full initialization because the redundant data blocks are written by file-system creation and by file-write and update. After a disk failure, any legal file-system operation can only access blocks that have been properly written (so the redundant blocks are in place). The advantage of a full initialization for Mirror-RAID-1 and RAID-10 is that the Raid-Group operation Begin Sync Check Scan of all Drives should find zero problems, or a very small number actually related to disk-drive issues. (Without this, a Sync Check Scan will find a very significant fraction of blocks, never written, mis-matched that should be duplicates.) What does not work with RAID-5 and RAID-6 -- the checksum blocks do not become initialized by ongoing file-system operation. It seems that writing a block within such a file-system will also update the associated checksum block or blocks based on the previous and new values of the block to be written, with the assumption that the checksum blocks were properly set. If they are not, the result does not converge toward full protection. Carefully updating a wrong checksum block produces more junk. The Initialize with full synchronization checkbox will be ignored for those Raid-Group types without redundancy. An Add Swapfile to Raid-Group option may be available, and if checked, will cause a swapfile to be created as part of the Raid-Group. A swapfile is a part of the memory management of many modern operating systems, and is used under high load to move apparently unneeded data out of RAM. Historically, ZipZerver have been created without swap space or swap files, and for a suitably modern system with oodles of memory and a modest load, a swapfile should not be necessary. This option is included here for experimental reasons based on potentially large load presented to underpowered hardware. If in doubt here, leave this off. The File-System Type can be either ext3 (which has been the type traditionally used by a ZipZerver), or ext4, which is a new implementation, is similar in design and use, and which can use much larger space for a larger file-system. ext3 is limited to 16TB (32-bit pointers locating 4096-byte file-system blocks). ext4 uses larger pointers, and thus can support much larger file-systems. The submit-button marked Check Raid Group Selections leads to the next step. The selected values are checked for consistency (e.g. too few disks selected for a Mirror group), and if no problems are found, a summary of what would be done is given (e.g. "Create a Raid-Group of approximately 400000MB...") with another submit-button marked Create Raid Group Now. You can initiate creation of the Raid-Group, or you can change any of your selections and re-run the checks with the Check Raid Group Selections submit-button. Success from the Create Raid Group Now operation means that Raid-Group construction has begun, or at least that it did not fail on step-0. The current state of the create operation can be checked on the main Raid Group page. For some number of seconds, it may appear that there is almost no disk activity, as partition tables are changed on each disk. The build of File-System structures will be marked by furious I/O activity that can last a half-hour (depending on size) before the File-System is on-line and available for use. If this is a Raid-Group with redundancy and Initialize with full synchronization has been selected, then the Raid-Resync gets into full swing in which duplicate blocks are written or checksums computed and stored. A RAID-5 made from 500GB disks can be in full synchronization after about 5 hours (around 100 GB/hour being written). The File-System is usable during the Raid-Resync (with some performance penalty) to create shares and store data, and all data that is written to the File-System will have full redundancy, but a Sync Check operation would find Mismatch Count inconsistencies unless the synchronization has run to completion. The relationship between Raid-Group level, redundancy, File-System size and performance --
Notes about the table --
Zerver Pair - Link RAID Group and File System between two ZipZerversEalier versions of this firmware had a feature which would link the Raid-Groups on two separate systems, with one Raid-Group in the Primary role, holding a File-System which is Available mounted Online, which would have Shares defined, and clients connected reading and writing data. Write operations to the File-System within this Raid-Group would also be sent to the ZipZerver with the Raid-Group in the Secondary role. Updates to various projects within the firmware made it impractical to maintain this feature. Perhaps it will re-appear in a future version.
List of SharesA Share definition is a name added to a directory within a Raid-Group File-System so that the directory, its files and contained sub-directories can be accessed by client systems. The Share definition holds the restrictions on which protocols may be used, which users may access, and what kinds of operations, read-write or read-only, are allowed. A system might have one Share definition, or many. Within File-system space, multiple shares could be disjoint or could overlap in various ways. The Share Name and Description columns are the externally-visible identifiers for a share. The Path column defines the location of the share-point in terms of the Raid-Group name (or Raid-Group mount point) and directory-path within that File-System. This is the column which will reveal if one share is actually a sub-set of another share. Two shares could actually have the same share-point -- same Raid-Group and same directory-path. For example, one share might be Public and Read-Only, and another could be Read-Write to a Restricted set of User-Accounts. For Attributes of a share, the most significant is first -- either ok or n/a. The value ok means that the Raid-Group exists, is Available mounted Online, and the directory-path for the share definition exists and is usable as a directory. Access to the share should not fail for technical reasons (but access might be denied for Permission reasons). The value n/a is for "no access", is generally highlight in yellow, and means
Other Attributes -- A public share has the attribute p -- anyone with access to the ZipZerver (for a given protocol) can access the share. The absence of "p" is for a non-public, restricted share -- access is allowed only for those User Accounts selected in the Permission list. A read-only share has the attribute r -- modifications to the share files and directories are not allowed. The absence of "r" is for a read-write share, modifications to the share files and directories are allowed (provided the share is public or the User Account has access permission to the share). A hidden share has the attribute h -- for several protocols, a request for a list of available shares will omit hidden shares from the list. Access might be allowed, but first the client needs to know that the share exists. This can help protect data from accidental access by users who click on every visible link, but does not really increase the security of the system since no user is actually denied access. The absence of "h" means that the share-name is visible. Windows file-systems maintain a number of properties about each file which do not have any correspondence in Linux, including "Archive-needed", "System-file" and "Hidden" (which should not be confused with the share-name property of the same name). If the ZipZerver attribute a appears, then these Windows file-attributes will be maintained (encoded using the Linux "execute" permission bits), which is probably a good choice for Windows clients. The absence of the "a" means that Linux "execute" permission will be maintained for each file, which is way better for Linux or Unix clients using NFS. For other protocols, using the Windows attributes is generally not harmful. The column Protocols shows which protocols have been selected to access the share and which protocols cannot be used for the share. Note that these selections and this display are independent of whether a given protocol has been enabled and is currently running, as set with the Network admin web pages. The link marked Permissions and the link using the name in the Share Name column go to the same admin web page, in which the share properties can be changed, and the list of User Accounts for the share can be checked and adjusted. To see the access permissions for all User-Accounts for all Shares, check the admin web page Security -> Share-User-Permission.
Create a ShareThe New Share Name is limited to 24 characters in length, and can be created using a wide range of characters, letters, digits, punctuation, and space. The name will be case in-sensitive in some contexts, such as Windows, and case sensitive in others. Characters not allowed in a Share Name --
For the characters that can be used, Share Names might be created that cannot be handled by some clients for some protocols. During testing, a share named "A!B@C$D%E^F&G _-+H" gets heavy use with Windows, HTTP, SmartMirror, FTP, and NFS within a test script. Some possible Share Names are dis-allowed, because of conflict with other parts of a ZipZerver.
The Share Description field is visible in a number of contexts in which a list of shares is given, and would be used by clients to help recognize the Share. The maximum length is 48 characters. The Share Attributes describe the fundamental properties of the Share. For a Public Share anyone with access to the ZipZerver (for a given protocol) can access the share. For a Non-Public Share or Restricted Share, access is allowed only for those User Accounts selected in the users list on the Permissions admin web page for the result Share. A Read-Write Share allows modifications to the share files and directories, provided that the share is public or the User Account has access permission to the share. Within a share which is Read-Only, modifications to the share files and directories are not allowed. For a Hidden Share, for several protocols, a request for a list of available shares will omit hidden shares from the list. Access might be allowed, but first the client needs to know that the share exists. This can help protect data from accidental access by users who click on every visible link, but does not really increase the security of the system since no user is actually denied access. For non-Hidden Shares, the Share Name will be visible, and will appear on various protocol lists of Shares. If Encode Windows File Attributes is selected, Windows file-system properties "Archive-needed", "System-file" and "Hidden" (which should not be confused with the share-name property of the same name) will be maintained for each file, encoding the values using Linux "execute" permission bits. This is useful for Shares in which Windows programs expect to be able to set the "Archive-needed" bit, and have it stick. This can be confusing if the Share actually contains Linux or Unix executables when viewed by a Windows program, because all of those files will be reported as "Hidden" (or not reported at all). The alternative is to Use Linux/Unix File Attributes, and is important if the share will be used for NFS access by Linux or Unix clients that need the execute-permission bits to operate properly. The various network Protocols can be enabled or dis-allowed for the Share. The process to define, and perhaps create, the Directory for the Share is done in several steps. If you happen to know the full-path desired for the share, and that directory already exists, you can select Full Path Name for Directory and enter the path directly. Generally you will start with Select Raid-Group, and hit the Check Share-Create Values submit-button. Once a Raid-Group has been selected, the re-display of the Create-Share web page will show any directories that already exist, with a field that can be selected and used to create a new directory. Each time that the Check Share-Create Values submit-button is hit, focus moves up and down the tree of directories within the Raid-Group. If there are no problems detected, an additional submit-button is offered -- Create Share Now, which will use the Share Name, Attributes, Protocols, and Directory for the Share values to Create the Share Definition. If you return to the List Shares admin web page, the new share will be shown with links on the name of the share, and in a column named Permissions. Either of these links will select the web page Edit Share Attributes and Select Users for Share Access for the share. This is where the list of Users with share permission is set. Many other Attributes of the share can be adjusted also. The set of users for a Share (or for multiple Shares) can also be set on web-page Security -> Share-User-Permission.
Edit Share Attributes and Select Users for Share AccessUpdated 2023-Jan-07 for NFS Client Addr/NamesA number of properties of a Share can be altered at any time with the Storage -> Shares -> Share-Name edit page, which can also be reached as Storage -> Shares -> Permissions for each share. Two properties that cannot be altered are Share Name and Directory for the Share. You have to create another Share definition if either of these values are unsatisfactory. The Share Description field is visible for several protocols which provide a list of Shares. The Attributes that can be altered -- For a Public Share anyone with access to the ZipZerver (for a given protocol) can access the share. For a Non-Public Share or Restricted Share, access is allowed only for those User Accounts selected in the users list below. A Read-Write Share allows modifications to the share files and directories, provided that the share is public or the User Account has access permission to the share. Within a share with Read-Only Access, modifications to the share files and directories are not allowed. For Share Name Visible, the Share Name will appear on various protocol lists of Shares. For a Hidden Share, for several protocols, a request for a list of available shares will omit hidden shares from the list. Access might be allowed, but first the client needs to know that the share exists. This can help protect data from accidental access by users who click on every visible link, but does not really increase the security of the system since no user is actually denied access. If Encode Windows File Attributes is selected, Windows file-system properties "Archive-needed", "System-file" and "Hidden" (which should not be confused with the share-name property of the same name) will be maintained for each file, encoding the values using Linux "execute" permission bits. This is useful for Shares in which Windows programs expect to be able to set the "Archive-needed" bit, and have it stick. This can be confusing if the Share actually contains Linux or Unix executables when viewed by a Windows program, because all of those files will be reported as "Hidden" (or not reported at all). The value Use Linux/Unix File Attributes is important if the share will be used for NFS access by Linux or Unix clients that need the execute-permission bits to operate properly. For the Protocols listed, each protocol can be allowed for the share or disallowed. These values are security selections, and might show, for example, that a share is not to be accessed using FTP. That selection remains, independent of whether the FTProtocol is enabled or disabled within the Network controls. Note that a Protocol that was not available when the share was created, SmartMirror Encrypted (rsync_ssh) in particular, is likely to be off until enabled. Also note that Encrypted FTP over ssh is not listed here, since its controls do not allow for enable/dis-able for each individual share. That protocol is either available for all shares, or for none. The list of Users shows both User-Accounts defined on the ZipZerver and those names known through Active Directory. Those with permission to access the share have a select-mark in a checkbox. If this is for a Non-Public Share, only those selected Users would be allowed access to the share. If this is for a Public Share, then all Users would be allowed access to the share, and the list is maintained in case the share becomes Non-Public at some time in the future. Below the list of known users, there may be a section Additional names found showing User-Account-like or Windows-DOMAIN\Names that are listed as having permission, but that are not in the list of known names. This might occur if one or more Windows-DOMAIN are in-accessible, so that a set of DOMAIN\Names are temporarily unknown. There could be circumstances in which a ZipZerver-local User-Account was deleted, but that name was not removed from the permission list for a share. These names can be kept, by leaving the select-mark in the checkbox, or removed from the permission list, by removing the select-mark in the checkbox. Permission or restriction for Unix-NFS clients operates differently from the other protocols. For Unix-NFS, the client is a system, rather than a userid. With earlier versions of this firmware, any Unix, Linux, or NFS-capable system in the world would be able to access any share which had Unix-NFS enabled, with the understanding that the ZipZerver would be behind a firewall which would prevent most of the world from actually connecting. And that if a ZipZerver were not behind a firewall and exposed to access from anywhere on the internet, then Unix-NFS should be Off. With Version 1.2.15, there are now controls to allow some Unix-NFS clients access, and to deny all others. The field NFS Client Addr/Names can specify one or more systems to allow Unix-NFS access. The entries are SPACE-SEPARATED list of IPv4 or IPv6 addresses, IPv4/masksize or IPv6/masksize subnets, or system names (either short or fully-qualified DNS-format). For names, it may work to use name-like values with characters "?" or "*" which represent any single valid character or any substring. Whether any name works, with or without wild-card characters, depends on the network's name-resolve mechanisms. After the "Save and Apply ..." submit button is hit, if there are syntax errors or problems with name resolution, error messages will be displayed. Other entries, without errors, will be applied. The link NFS Status More Details Here goes directly to the Network -> Unix admin web page which will show which exports were successful and show the full output, with error messages, for the exportfs operation. Note that you can see error messages caused by improper entries with NFS Client Addr/Names for other shares. The default value for this field is the set of IPv4/masksize and IPv6/masksize that represents the local network, as understood by the set of physical network connections. To allow Unix-NFS access from any system, use the value "*" by itself. If you try to make this field empty, the default value, for the local network, will be filled in. This web-page shows the permission list for all users for this one share. There is another admin web page at Security -> Share-User-Permission, with a matrix showing permission for all users for all shares. That may be more comprehensible, and can be used to change permissions for all shares at one time, but it does not handle changes of any other Share attributes, and does not show the NFS Client Addr/Names values. All of these values can be changed, and put into effect by hitting the Save and Apply Security/Permission/Attribute Choices Now submit-button. The new values take effect immediately for new clients, or previous clients re-establishing a connection. There can be long-life connections between a client and a ZipZerver, and those would continue to exist and run with Attributes and Permissions as existed when the connection was established. Removing a User-Account from a permission-list, with Save and Apply, will not disconnect a previously-connected User.
Advanced Use of NFS Client Addr/Names -- The handling of the NFS Client Addr/Names field is done by a program named exportfs, using the format for a file named /etc/exports. There are additional things that can be done. This is not recommended, however, unless you have expert advice.
Immediately after an entry,
IPv4 or IPv6 address,
IPv4/masksize or IPv6/masksize subnetwork,
name,
or name using "?" or "*" wild-card characters,
you may place a (parenthesis-list) of one or more NFS export options
(with no space after the entry and before the parenthesis).
Example --
It looks like one could over-ride the Read-Only Attribute of a share selectively --
Among the many reasons to avoid experimenting with this (mis?)feature -- a mis-spelling of any option, with error message "unknown keyword", ends processing of the export operation. This can have the effect of un-exporting other Unix-NFS shares. The link NFS Status More Details Here goes directly to the Network -> Unix admin web page which will show the detailed non-default options for each allowed export and client. The Help link on that page has some notes about a few of the export options that might be specified.
Delete One or More SharesThe Delete Share admin web page has most of the same data fields as the List of Shares web page.Note that the Delete Share operation does not delete any of the data. The directory-path in the Share-Definition, if any, will remain, and all contained files and sub-directories will be untouched. Only the Share-Definition -- including Attributes, Protocols, and list of Users -- is removed. The data may no longer be visible using the deleted Share Name, but the existence of the data is not affected. Select one or more Share-Definitions by marking the checkbox by the Share Name and hit the Share Definition Delete Now submit-button. Share-Definitions will be deleted, and configuration data for all protocols will be updated. Subsequent client connections will use the updated, shorter, list of shares. (Note that current connections, established with the earlier share list, can continue.) There is even a use for selecting zero Share Names -- the re-evaluation of configuration data for all protocols is still done. There could be some odd conditions in which ZipZerver config files have been updated by some non-standard means, such as using FTP to the /flash directory. Select zero Share Name entries and hit the Share Definition Delete Now submit-button to force the re-evaluation, to ensure that the configuration data for all protocols is consistent and up-to-date.
Disk Usage Report for each ShareUpdated 2019-July-25"No matter how big the disk, it will be mostly full in three months." -- attributed to Murphy as a corrollary to Murphy's Law. Use this page to create a report of disk-usage for each share. Details can be displayed for just the share itself, or for the disk-space used in sub-directories up to several levels deep. The report can be scheduled to run once a week, or can be initiated when needed. The mechanism involves checking every directory and sub-directory reachable from a share and checking the file-size for every file. If there are many, many files this process might take minutes or longer. Thus the advantage of a scheduled task, so that the report can be ready at a convenient time. The field Max Depth shown in report provides control over the level of sub-directory displayed. Value 0 will display just the directory for the share. Values 1, 2, 3... will provide disk-space usage that many sub-directories deep. It is hoped that this can help answer the question, "If the share is using X KB of disk-space, where is it?" Setting this value lower does not make the disk-usage scan go faster, since all of the files in all of the directories all of the way down must be checked. This only controls how much is displayed. For technical reasons, the disk-usage scan is done in directory-slot order, not in alphabetical order. Results of 2-3 levels of directory are easier to interpret when the listing is alphabetical. A sort is done so that, mostly, the sizes of sub-directories are immediately followed by the size of the containing directory. Because this system has limited resources, the size of the list to be sorted is limited (around 2000 entries). If the display for a share hits this limit, that share's results are abandoned and processing continues with the next share. If there are too many entries to sort when Max Depth shown is 3 (for example), try running the report again with a value of 2. All of the files in all of the directories must still be checked, but the size of the display list will be shorter. The selection Enable, Run Automatically by Start Time Choices will cause the Start Time values to be interpreted, and the DiskUsage Task will be scheduled to run automatically once a week. Disable, Do Not Run Automatically will prevent an automatic run. The submit-button Update status values above will just re-display the page, including any Status values above or the DiskUsage report below that have changed. The submit-button Save Auto Run Start Time Values will save the Max Depth, Enable/Disable, Hour, Minute, and Day-of-Week values, changing the automatic task schedule ("crontab") as appropriate. The submit-button Save Start Time Values and Start Report Program Now will save the configure values, and will also initiate a DiskUsage Task. The selection Enable or Disable makes no difference here, the Task will start either way. The Status line for most recent start should be updated immediately, but there might not be any other results to display for seconds or longer. Recall that the full report on a system with many files can take minutes or longer before the report is complete. Because a DiskUsage Task can be started two different ways (by schedule or by manual initiation), it is possible for two (or more) copies of the program to collide. A file lock is used on the result file so that if a DiskUsage Task begins and discovers that another Task is already writing the result report file, it ends without doing any damage, mostly silently (leaving messages in the Status -> Current Details syslog). The result file, empty, complete, or partial, is displayed as part of this web-page. A less-cluttered display can be found with the Result File link or by direct location within the hidden share /flash/ as file /flash/diskusage.txt. The result file shows the start and end-time of the report. Each share name is listed, followed by the (sub-directories and) directory for the share. The directory name is the one used when the Share-Definition was created, and might not be the same as the name of the share. Should the total DiskUsage for all Shares within a Raid-Group match the File System Used MB shown on the List Raid Groups page?
List of UsersThe List of Users admin web page shows userid which are local to the ZipZerver, as created with web page Security -> Create User. For a list of userid that also includes Windows "Active Directory" DOMAINNAME\Userid, see the web page Security -> Share-User-Permission.The display is sorted under the userid column, with each name a link leading to a Change User Atrributes or Pass Phrase web page for the userid. The column Full Name or Description shows the descriptive data, if present. Several operations that involve use of ssh-Secure-Shell for encryption can be individually allowed or denied. The column SMirror/rsync-ssh shows if the userid can be used for an Encrypted SmartMirror task or other use of program rsync over ssh protocol. Similarly, FTP-ssh shows if an FTP session over ssh encryption is allowed for the userid. The column SVN-ssh shows if SVN-Subversion access is allowed for the userid. The cmdline ssh column shows which userid have been granted command-line ssh login privilege. The value "yes" is given for those userid which may use SSH-Secure-Shell protocol to login to a command-line prompt. This can be used for certain kinds of system diagnosis and maintenance, but is expected to be seldom used. No value is shown for ordinary userid. The column web-admin shows which userid have been granted admin web page access privilege. The value "yes" appears for those userid, in addition to admin, which may access the admin web pages. No value is shown for ordinary userid. The column RO-web-admin shows which userid have been granted read-only admin web page access privilege. The value "yes" appears for those userid which may access the admin web pages for examination or audit, but which cannot use a submit-operation to change any values. No value is shown for ordinary userid.
The userid admin is not shown in the list of userid. To change its Pass Phrase, use the System -> General admin web page. Other built-in userid names, such as root and public are not shown, since proper system operation requires that those userid be left alone.
Create New UserUpdated 2022-Jan-28 for SVN-sshThe userid created with admin web page Create New User are local to the ZipZerver, and the Pass Phrase should be a secret between it and the userid. By using the New Passphrase web-page, the user can change their passphrase so that it is no longer shared with the admin which created it. This is in contrast with userid that are part of a Windows "Active Directory" Domain in which a DOMAINNAME\Userid is created at a Windows Domain Controller, can be known and used by any number of machines which are part of that Domain (or part of another Domain with a suitable trust relationship), and the Pass Phrase is a secret between the DOMAINNAME\Userid individual and the Domain Controller. The userid field itself is the short, unique, perhaps cryptic, identifier that would be considered "machine friendly." You can use "a" - "z", "A" - "Z", "0" - "9", "$", underscore-"_", and dash-"-", up to 31 characters in length. (The space character can also be used, for some very limited purposes.) By tradition, Linux/Unix userid use lower-case letters, but the wider range of userid characters work, provided they are used consistently. Note that userid of all digits are not allowed, because it would be ambiguous (in some contexts) whether the name "123" was intended, or the userid whose assigned user-number was 123. The field Full Name or Description is intended for more human-recognizable text. This may be left empty, does not need to be unique, can be up to 120 characters, with a wide range of values allowed but not colon-":", double-quote-'"', or single-quote-"'". Each of these limitations is for various internal technical reasons, and the last is disappointing, since there has been a Mr. D'Ascoli on staff whose correct name cannot be entered. The field is named Pass Phrase/Word to encourage use of a phrase, rather than a single word or word-like clump. The Pass Phrase is limited to 120 characters, and the only character rejected is "new-line" (created with the "Enter" key), but that character (and "tab") cannot be passed as data from many browsers anyway. Several operations that involve use of ssh-Secure-Shell for encryption can be individually allowed or denied. These operations need special treatment since the standard userid creation does not allow any command-line login. Encrypted SmartMirror/rsync over ssh will allow (or deny) the userid for an Encrypted SmartMirror task or other use of program rsync over ssh protocol. Similarly, Encrypted FTP over ssh will allow (or deny) an FTP session over ssh encryption. Encrypted SVN over ssh will allow the userid access to an SVN-Subversion repository, if configured. The command-line ssh login checkbox, if granted, would allow that userid to login, using protocol ssh-Secure-Shell, to a command-line prompt. This can be used for certain kinds of system diagnosis and maintenance, but is expected to be seldom used. The ssh-Secure-Shell operations can be allowed or denied individually, in any combination. The admin web page access checkbox sets admin privilege, permission so that the newly-created userid will be able to use the admin web pages (and thus create additional userids, as well as all the other admin operations). The read-only admin web page access checkbox sets partial admin privilege which allows admin web-pages to be examined but does not allow any Submit-operation that would change anything. If both admin web page options are selected, the read-write admin privilege would dominate. The Create New Userid Now submit-button initiates the syntax checks, and creates the userid in two steps -- first create the userid, then set the Pass Phrase so that the userid is usable for each protocol. Because the Pass Phrase is stored in different format for use by Windows SMB/CIFS, SmartMirror, and for all other protocols, there could be some odd failures in which the Pass Phrase is set properly and will work for some protocols, but not for others. Any failure message should be considered carefully -- after partial success (i.e. partial failure), the userid should probably be deleted. Note that if the userid already exists, this operation will work as an update, and change the previously existing Full Name, permissions, and Pass Phrase, rather than reject the operation. This behavior is the result of using a small number of simple mechanisms for userid management, is handy for system test scripts which can create and re-create userids many times, but does seem to violate the principle of least astonishment. Some possible userid values are dis-allowed due to conflict with pre-defined, special purpose userid -- root, bin, nobody, sshd, and public. The userid admin should be manipulated from the System -> General web page. Although intended to be simple, this userid mechanism has been tested with up to 400 userid.
Change User Atrributes or Pass PhraseUpdated 2022-Jan-28 for SVN-sshThe admin web page Change User Attributes or Pass Phrase can either be used to change just the User Account Attributes -- the Description field and special permission values -- or can be used to change just the Pass Phrase, or can update both. The userid is the unique identifier that specifies the User Account. Only userid which are local to the ZipZerver can be updated, as created with the Security -> Create User web page. The Full Name or Description field is the more human-recognizable text, and can be used for any descriptive purposes, or left empty. It may be up to 120 characters, but may not use characters colon-":", double-quote-'"', or single-quote-"'". Several operations that involve use of ssh-Secure-Shell for encryption can be individually allowed or denied. These operations need special treatment since the standard userid creation does not allow any command-line login. Encrypted SmartMirror/rsync over ssh will allow (or deny) the userid for an Encrypted SmartMirror task or other use of program rsync over ssh protocol. Similarly, Encrypted FTP over ssh will allow (or deny) an FTP session over ssh encryption. Encrypted SVN over ssh can allow the userid access to an SVN-Subversion Repository, if one is configured. The command-line ssh login checkbox, if granted, would allow that userid to login, using protocol ssh-Secure-Shell, to a command-line prompt. This can be used for certain kinds of system diagnosis and maintenance, but is expected to be seldom used. The ssh-Secure-Shell operations can be allowed or denied individually, in any combination. The admin web page access checkbox sets admin privilege, permission so that the userid will be able to use the admin web pages (and thus create additional userids, as well as all the other admin operations). The read-only admin web page access checkbox sets partial admin privilege which allows admin web-pages to be examined but does not allow any Submit-operation that would change anything. If both admin web page options are selected, the read-write admin privilege would dominate. The Update Userid Information Now submit-button can be used to change the above elements, and leave the userid Pass Phrase unchanged. The Pass Phrase/Word field can be up to 120 characters, and if only the Pass Phrase needs to be changed, the Set Userid Pass Phrase Now submit-button can be used. If the Pass Phrase is to be updated, and one or more of the elements from the Userid Information section must also be updated, the Change Both Userid Info and Pass Phrase Now submit-button can be used.
Delete One or More UsersThe Delete One or More Users web page looks very similar to the List of Users page, with the addition of checkboxes for each userid to select to be deleted. These are userid which are local to the ZipZerver, created with the Security -> Create User page. As userid are deleted, any Share Permission granted to that userid for any Share is also removed.
Current Permission for All Shares for All UsersUpdated 2022-Jan-28 for SVN-sshThe admin web page Current Permission for All Shares for All Users is intended to present almost all Share-Permission information for all userid for all Shares, and allow set and update for multiple Shares and multiple userid in one step. The column Userid includes userid created on the ZipZerver using the Security -> Create User web page, and Windows "Active Directory" DOMAINNAME\Userid from the Windows Domain that was joined, and any Domains with a suitable trust relationship. The Shares are listed by name as a link to the web page that shows and allows edit of the Share properties. For each userid and each Share, the current permission is shown, and a checkbox is available to change it. For a given userid, an ordinary read-write share will show
If Share Permission has been granted to a Public Share, the "RW" and "RO" are displayed, rather than "Pub" or "Pro", so that the explicit permission is visible. That permission would become significant if the attributes are changed, and the Share is no longer Public. After review of these values, any number can be changed using the checkboxes, and Update Share * User Permissions Now submit-button can be used to apply all changes in one step. The display is intended to make clear the relationships between userid and Shares. The set and update for multiple Shares has the advantage of all changes being applied at the same time. If the Share definitions have been made so that no two Shares overlap, then this display of permissions is a complete display of who can access what. It is incomplete, however, if any Share is a subset of another -- the Share-point directory of one is a subdirectory of the Share-point of the other. For this case, userid with permission to one share have partial access (through the subset) or complete access (through the superset) to the other. These relationships are not shown on this display. See the admin web page Storage -> Shares, and examine the Path column to see if there are any subset-superset relationships. An additional overview of which userid have what capabilities is shown in the columns labeled Other Operations. The column Smirror/rsync-ssh shows which userid can be used with Encrypted-SmartMirror tasks, which can be widely allowed (and which can be adjusted en-masse on this web page). The column FTP-ssh shows those allowed to use Encrypted-FTP over ssh, and those should probably be more cautiously allowed, since most of the ZipZerver security controls do not apply -- if this protocol is enabled, then every userid that is allowed to use it has access to every share. The column SVN-ssh shows if access to an SVN-Subversion Repository is allowed, if one has been created. There is more info at Subversion Use within ZipZerver The column cmdline ssh indicates which userid, if any, would be allowed to login and get a command-line prompt, to run any command that would work for a non-priviledged userid. This would be useful for some kinds of system diagnosis or maintenance, applied only to a short-lifetime userid created just for that purpose. The userids marked in column web-admin can perform all system operations through the admin web pages, and might have a longer lifetime if someone is the back-up administrator (and is a much better approach than sharing the password for the built-in admin user). The column marked as RO-web-admin shows which userids have read-only access to the admin web-pages, which may be useful for audit or demo purposes. Such a userid will be able to follow links within the admin web-pages, but would not be allowed (unless it has web-admin privilege also) to submit any changes to the system. The intent of this display is to give an over-view of all userid, and show what system resources are accessible to each, and to allow major or minor changes throughout.
SmartMirror Task Current StatusUpdated 2017-Feb-08The web page SmartMirror Task Current Status shows information about the most-recently started SmartMirror client task, either currently-running or ended. Recent Task Start/Stop is the time-stamp on a file generally written only when a SmartMirror task is started. The Task Name is shown, if known. If Most Recent Task Status matches Date & Time Now, then the SmartMirror Task was still running when the page was displayed, and there should be some progress-log text. If the Task Status time-stamp is in the past, then the SmartMirror task had already ended (when the page was displayed), and there should be progress-log text that looks like a task ending result. Start/Stop and Task Status will be "(none)" if no SmartMirror client Task has run since system startup. If there have been any SmartMirror client Tasks, Most Recent Task PID will show a PID-Process-ID integer value of the most-recent task, along with a submit button Force Terminate this Task. This can be used if a SmartMirror task appears to be stuck (perhaps with network problems or preparation of an unexpectedly large file list at the remote end) or the result of the task is undesired. If the SmartMirror task is still running when Force Terminate is hit, the task will be ended with a cryptic error message. If the task is part of a chain of tasks, the chain will be broken and the next task will not be run. If the SmartMirror task has already ended (and perhaps the next task in a chain of tasks has begun), there will be no effect, and any currently-running tasks will continue.
Schedule for List of Defined SmartMirror TasksA SmartMirror Task defines a client operation to run on this ZipZerver, interact with a SmartMirror server running on another ZipZerver, to transfer the contents of a directory on one system so that it will be matched to a directory on the other. Communication bandwidth will be used efficiently, transferring only the files needed, and sometimes transferring only the parts of files that are needed. Web page Schedule for List of Defined SmartMirror Tasks has an entry for each Task, with the Task Name, source and destination, scheduling values, and the name for the Next Task, if part of a chain of Tasks. To check or to edit complete details, use the link on the Task Name. There can be as many as 360 scheduled tasks. For the Source and Destination columns, for each entry, one will be marked as Local: with a Share name, and the other will be identified with an IP-Address or DNS-name, along with a Share name to be found on that Remote Server. A directory within the Share, Local or Remote, can also be shown. This is the representation for Direction choices Pull from Remote Server and Push to Remote Server. If the tag ssh: appears before the IP-address or DNS-name, then the connection will be encrypted using ssh-Secure-Shell. A column next to Source may say newer: to represent Direction choices Pull from Remote, Newer Files Only and Push to Remote, Newer Files Only. The column next to Source may say rename: with a (not really) Remote Source of "127.0.0.1" to show Direction Pull by Directory Rename, along with del: next to the Local Destination for Delete Local and Pull by Directory Rename. The IP-Address "127.0.0.1" is a special case, known as "localhost", and refers not to a machine which is actually Remote, but to the ZipZerver itself. If the Task has been scheduled to run once in a day, then the Start Time field will display as HOUR:MINUTE, such as 23:30 for 11:30 PM. If the Task is to run multiple times in a day, the HOUR field can be ALL, to select all the hours from 00 to 23, or can be DAY, to select every hour from 07 to 19, 7 AM to 7 PM. A Start Time field of DAY:30 would appear for a task to run every hour from 7:30 AM to 7:30 PM. The MINUTE field may appear as 15,45 to represent twice in an hour. The value 0,10,... means every ten minutes within an hour. The Start Time value of ALL:15,45 means the task is scheduled twice an hour throughout the night and day. The value 03:0,10,... would be shown for a Task to be scheduled 6 times, from 03:00 AM to 03:50 AM. An immediate, practical use for such a selection does not come to mind. The display of SmartMirror Tasks is sorted in a way intended to make the schedule comprehensible, to show the Task relationships, and to make a Task findable based on Frequency, with Weekly Tasks shown ahead of Monthly, with MTWTF Tasks ahead of Monday-only, with Monday Tasks ahead of Tuesday, with Monthly Tasks sorted by DayOfMonth, and Start Time value sorting for tasks with the same Frequency. (The Enable/Disable Status is not used to determine the display order, so you can find a Task by its Frequency scheduling, whether Enabled or not.) For Tasks that are part of a chain shown with Next Task names (and whose position is not otherwise determined), an attempt has been made to display in chain-execution order. A chain of tasks with a recognizable begin and end will display fairly well. (For stress testing with a never-ending loop, there is no recognizable last-task.) If there is a Task with two predecessors, it may be displayed in an order that makes sense for one of them. And if nothing else, the Tasks will be sorted by Task Name.
Create or Edit a SmartMirror TaskUpdated 2018-April-01, primarily for Encryption.The admin web page Create or Edit a SmartMirror Task can specify or alter all of the details of a SmartMirror task, to define a client operation to run on this ZipZerver, interact with a SmartMirror server running on another ZipZerver, to transfer the contents of a directory on one system so that it will be matched to a directory on the other. Communication bandwidth will be used efficiently, transferring only the files needed, and sometimes transferring only the parts of files that are needed. You can have up to 360 SmartMirror tasks. SmartMirror Task Settings The Task Name identifies the task on the Task Schedule list, appears in status and error messages, and can be used as the target name when building a chain of tasks. It can have up to 30 alphanumeric characters. You can add a little bit more description in the Comment field. This value is only shown with this Edit Task page. The comment can contain up to 45 alphanumeric characters, including the space. Select the Direction that the data in the SmartMirror backup is to go from the list of radio buttons. Is the data to be pushed from this system to the remote system, or is the data to be pulled from the remote system to the ZipZerver? The Direction choice also selects what happens at the destination end. The basic choices Pull From Remote Server and Push to Remote Server will duplicate exactly the source directory at the destination, and will remove any extraneous files at the destination that do not exist at the source. Use special caution with these choices. If the Destination Share or Path in Share are close, but not correct, there can be massive file delete from that Destination directory to make it match the Source Directory. Other Direction choices, Pull From Remote, Newer Files Only, and Push to Remote, Newer Files Only, will send files that have a newer time-stamp on the source system, or which do not exist on the destination system. For these Update choices, a file on the destination system with a newer timestamp is not overwritten, and no files are removed from the destination system. These choices are used when both source and destination might have valuable data to be maintained (rather than the source system holding the real data, and the only role of the destination is backup). Additional choices for Direction are very specialized and were added to be part of a chain of tasks used to maintain a time-sequence of backup directories. For selection choices Pull by Directory Rename and Delete Local and Pull by Directory Rename see further discussion below in section Directory Rename. Local Settings. Select the name of the share for which you want to schedule the SmartMirror backup from the Share drop-down list. If you want to narrow the SmartMirror backup to a folder within the share, enter the path to the folder in the field Path in Share. Use Unix/Linux path name rules -- Uppercase and lowercase letters must match exactly, and directories and folders are separated by a "/"-slash character. If used, this field must name a directory, not a file. A leading "/"-slant character is not necessary. For example, within a share named "ABC" might be a file with path-name of "ABC/Def/ghi/JKL/file.txt". To select the directory containing this file, select share "ABC", and set Path in Share to "Def/ghi/JKL" (without the quote marks). Remote Server Settings. The Remote Address may be identified by its IPv4- or IPv6-address or by its DNS-name. You must also provide the name of the Share on the Remote Server, the Path in Share if you want to backup a specific folder in the share, and the Login Name and Password for that Share on the Remote Server. Since the configuration and status of the Remote Server is not as well known (as the Local Server), the possible Share names are not listed as a drop-down list. However, a listing of Shares from that Remote Server can be obtained using the Check Remote Host submit-button, described further below. If the Remote Server uses a non-standard port number for SmartMirror operations, other than 873, as configured on that Remote ZipZerver Network -> SmartMirror web page at the SmartMirror Port Number field, there are two ways to specify. The newer, preferred way is to give the port-number in the AltPort field. An older syntax may also be used (for non-Encrypted SmartMirror tasks), by adding the port-number to the Remote Address field. If the Remote Server IPv4 address is 192.168.1.100 and the SmartMirror Port Number is 871, then the Remote Address field would be -- 192.168.1.100:871 Since IPv6 addresses are generally represented using ":"-colon characters, the address value can be put in square brackets -- [2606:2800:220:1:248:1893:25c8:1946]:871 Data Transfer Settings. Additional options can be selected for the scheduled SmartMirror backup. Selecting Encrypt Session will create a SmartMirror task which uses program ssh (Secure Shell) to communicate with the Remote ZipZerver, with everything encrypted. Without this, the communication is done in clear text and could be subject to eavesdropping by routers between the two systems. For an Encrypted task, a check is made to see if any public keys are known for the Remote Server, and if any are found, a hash-fingerprint will be displayed. There is much more to say about Encrypted SmartMirror tasks below, in Section Encryption. If you select Compress Data, the files from the source will be compressed as they are transferred. This takes additional processing time, but may be worthwhile if a lot of data must go through a limited speed link. You can choose to Create Backup Files in the Destination Share of any data files that are updated. The old version of the data file is renamed with a "~"-tilde character on the end, so that a file named "ab.cd.ef" is renamed as "ab.cd.ef~" before the new version of the file is written. If there already was a file with a "~"-tilde character, then it is renamed with two, to be "ab.cd.ef~~". A lot of back-up files can be created, with the filename with the single trailing "~"-tilde as the most recent before the current version. Excess back-up versions of a file are not removed automatically, and must be managed manually. If you select the Test Run Only option, the ZipZerver will run the SmartMirror task right up to the point where the data is actually transferred. The "SmartMirror Logs" will report whether the test operation was successful. This can be used to check the validity of the Remote Address, the Remote Share, the Path in Share, and the Remote Login Name and Password. The Other Parameters field allows for some specialized uses. The most widely useful value would be "-v" which would run the SmartMirror operation in verbose mode. Additional information is written to the SmartMirror Log file, including a list of files transferred and a list of any files deleted from the destination. A value of "-v -v" (but do not include the quote marks) would provide even more data in the Log file, which may be useful in understanding certain kinds of SmartMirror problems. The Other Parameters field can be used to achieve much more elaborate results using the full range of command-line parameters available for rsync, the program which does the heavy lifting for the SmartMirror feature. For example, to re-initialize an elaborate directory structure, duplicating the set of all directories and folders, copying none of the data files from the source, and removing any left-over data files at the destination, you can set the Other Parameters field to "--include */ --exclude * --delete-excluded" (but do not include the quote marks, and note the double "-"-minus character that introduces keyword parameters, rather than single-letter parameters). This feature is not recommended for use by the casual user. Other Options. After the Task Name and Comment fields, everything else defined above is related to the rsync task that will transfer the data -- the direction, location info, and options for the transfer. What follows are the options for SmartMirror, which schedules the task, sends e-mail alert messages for some conditions, and perhaps combines tasks into a chain of multiple tasks. Only one SmartMirror client is allowed to run at one time, so that the Current Task status display and SmartMirror Log output are unambiguous. Setting the Maximum Minutes to Wait field will allow the task to wait for other tasks to finish. If a task almost always completes within a few minutes, but might, under some conditions, run for hours over a slow link, then another task can be scheduled to run 10 minutes later and allowed to wait e.g. 720 minutes (12 hours). Alert messages can be sent by E-mail under any or all of a set of conditions -- at SmartMirror Task Start, if a task Cannot Start within Max Minutes, if a task Ends with a Problem, or when a task Ends with no Problem. A message will be sent to one or more recipients as configured on the System -> Alert Setup admin web page. Sending a message for normal ending, as well as abnormal conditions, from at least one regularly scheduled task can provide verification that E-mail alerts are being properly delivered. If there is a SmartMirror Task which should be followed by another only if the first task ends normally, then the Task Name of that second task can be given in the field Next Task to Run. This might occur if one Task pulls some data from one Remote System, and then a second Task should push that data to a second Remote System, provided the first transfer ended with no problem. Such a chain of Tasks can be of any length. Is it legal (and useful) to create a chain of Tasks that is a loop ? Yes, and this has value in creating system stress test load, and for running monitor tasks for which several times an hour scheduling is not responsive enough. In order to protect against un-intended never-ending loops, an upper-bound is enforced, currently 999 tasks in a chain. Scheduling. The scheduling of a SmartMirror Task can be toggled on and off by selecting the Enable or Disable values for Status. A task which is disabled does not run by its schedule selections, but can still run if it is named as the Next Task to Run of a Task which is run, or if the Save Info and Run the Task Now submit-button is used. For scheduling, indicate the Start Time you want this scheduled task to begin, using 24-hour clock notation. For example, if you want the scheduled task to begin at 11:30 PM, enter 23 in the first field and 30 in the second. On what days do you want this backup task to occur? You can indicate either a weekly or monthly cycle. If you select the Weekly radio button, you can choose from one to seven days on which to schedule the task. For example, you can schedule a task that backs up Share ABC every Tuesday and Friday at 3 o'clock in the morning. Choose all seven days if you want a backup done every day. If you select the Monthly radio button, select the day of the month you want the scheduled task to occur from the drop-down list. For example, if you want Share ABC to be backed up on the 15th of every month, select the number 15 from the list. Note that the value 31 means to run the task on the last day of any month which has 31 days (it does not mean "the last day of every month"). Tasks that must be run after a month has ended should be scheduled to run at the beginning of the following month. Special Start Time Hour, Minute Choices For specialized SmartMirror synchronize tasks, there are some additional choices for Start Time Hour and Minute. Instead of picking one of the hours from 00 to 23, you can choose ALL, which means that the task will be scheduled to run during every hour of the day. Another choice for the hour value is DAY, for which the task will be scheduled to run during every hour from 07 to 19, 7 AM to 7 PM. The choice for minute within an hour also has some extra values. Pick 15,45 so that if a task is to be run within a particular hour, it will be run twice, at 15 minutes and at 45 minutes after the hour. The value 0,10,... can be used to run a task every ten minutes. The value many can be selected to run a task even more often -- every four minutes. Update Task-Pair, Automatic Restore The combination of using an Update Task, selecting Pull From Remote, Newer Files Only, or Push to Remote, Newer Files Only, with scheduling once every hour or several times per hour, can be used to solve some difficult synchronization and back-up problems. Two ZipZerver with shares linked by a pair of Update Tasks (a Push and a Pull scheduled on one of them, or a Pull Task from the other scheduled on both of them) will do automatic backup, and will also do some kinds of automatic restore -- if a file is deleted accidentally, it will be restored from the other system by the next scheduled task-pair (e.g. within the hour). There is a certain amount of load-sharing possible -- users can map to either system based on load, speed, or availability, regardless of which system was the "original" and which the "back-up". This is not the same as real-time remote-mirror with file-locking, however. If a file is updated, separately, on both systems during the interval between scheduled Update Tasks, then the changes made to the file with the earlier time-stamp will be lost, the file with the later time-stamp will prevail, and that file will appear on both systems after the next Update Task-pair. Also, if a file is deleted deliberately from one system, it will be automatically restored from the other. If a file or directory is renamed deliberately on one system, another copy with the old name will automatically re-appear. Removing or renaming must be done on both systems before the Update Tasks run. For those who find the idea of automatic restore to be very pleasant, an additional caution -- the files need to be created and updated with sensible time-stamps. The ZipZerver can be configured to use a time-server for synchronization, but the time-stamp left on a file updated by a client using a ZipZerver is usually up to the client, not the ZipZerver. A PC-client with an ill-configured clock, or which failed to get a good setting from a time-server, might be creating time-stamps at some point in the future. Interesting symptoms can result from this condition. Directory Rename Direction choices Delete Local and Pull by Directory Rename and Pull by Directory Rename were created for use within an elaborate chain of SmartMirror tasks that can be used to manage a time-sequence of directories, such as when directories named "Week1", "Week2", etc. hold copies of a repository of files and folders as it existed one week ago, two weeks ago, etc. SmartMirror tasks using these direction selections require that the Remote Address of the "Remote Server Settings" be "127.0.0.1", which is a special IP address generally known as "localhost". On all systems, it addresses the "loopback" pseudo-device, and is thus an IP address by which a node can address itself. Therefore the entries under "Remote Server Settings" for Share and for Path in Share are not located on some remote system somewhere, but are on the same system with "Local Settings" entries for Share and Path in Share. In order for a directory-rename operation to work, the directories thus identified must be within the same Raid-Group File-System, and the expected use is that these will be two directories within a common container directory. A SmartMirror task defined with a Direction value of Delete Local and Pull by Directory Rename will perform several steps. (1) The directory identified under "Local Settings" by the entries for Share and for Path in Share will be deleted, including all subdirectories, folders, and files. (2) A rename will be done for the (local) directory identified under "Remote Server Settings" by entries for Share and Path in Share to use the name of the just-deleted directory given by "Local Settings". (3) An empty directory will be created using the just-abandoned name given by the "Remote Server Settings". For example, for an environment in which the state of a repository is captured every week and kept for four weeks, the Local Settings might select Share "ARCHIVE", Path in Share "Week4", within a task with Direction Delete Local and Pull by Directory Rename, which would delete directory "Week4" containing files more than four weeks old. If Remote Server Settings (with Remote Address of "127.0.0.1") selected Share "ARCHIVE" and Path in Share "Week3", then directory "Week3", holding files more than three weeks old, would be renamed and become directory "Week4". An empty directory named "Week3" is then created. The effect is that all of the files, folders, and subdirectories that had been within "Week3" are now within "Week4", files older than four weeks are gone, and this has been achieved without having to do a copy-operation. A SmartMirror task defined with a Direction value of Pull by Directory Rename differs from the above in that the target directory defined by "Local Settings" for Share and Path in Share must be empty (or not exist at all) -- a delete of any and all files, folders, and subdirectories is not done. For example, if directory "Week3" is known to be empty (as it would be after the previous example), then a SmartMirror task with Direction Pull by Directory Rename could have Local Settings of Share "ARCHIVE", Path in Share "Week3". If directory "Week2" had files which were more than two weeks old, then the task would have Remote Server Settings (with Remote Address of "127.0.0.1") of Share "ARCHIVE" and Path in Share of "Week2". The effect will be that the files and folders that had been within directory "Week2" will suddenly be found in directory "Week3". An empty directory named "Week2" will be created so that there can be a chain of these tasks. If a SmartMirror task with Direction Pull by Directory Rename encounters a non-empty target directory, then something has gone wrong and the task ends with error message "Directory is not empty" to be found in the syslog. Because both directories are to be found on the local system, the entries for Login Name and Password within "Remote Server Settings" are not used. Save Info Hitting the submit-button Save Info for Task will initiate a check for a certain amount of pickiness for the values specified. If all is well, a check is done for an internal identifier for this task. This is a value such as "[0000]" or "[0350]", and if one has not already been allocated (for a Create Task or Duplicate this Task operation), a value is chosen now. This can fail, if too many tasks already exist. Not all error checks are run when the task is saved. The value for Task Name is checked for duplicates, and the Next Task to Run value is checked for a match when the Task Schedule is displayed. The submit-button Save Info and Run the Task Now operation will attempt to run the task immediately, regardless of whether the Status is Enable or Disable, regardless of the other Scheduling values. As with any SmartMirror task, the result or current status of the run can be found on the SmartMirror Current Status web page. As mentioned above with the "Remote Server Settings", the submit-button Check Remote Host will use the Remote Address, either IP-address or DNS-address, to get a list of Shares. On success, the response from the Remote Address is shown, and indicates that the DNS-name, if any, can be resolved to an IP-address, that the IP-address is valid, at that IP-address is a ZipZerver with the SmartMirror server enabled (or another system with the rsync program listening and available on its port number, 873), and the non-hidden Share names are listed. There could be additional Share names (or rsync "module" names) that could be accessed, but which have the "hidden" attribute. What is not tested with the (non-Encrypted) Check Remote Host operation is the validity of the Login Name, Password, Share name, Path in Share, and permission. You can use the Test Run Only setting for that. The submit-button Delete this Task ... leads to an "are-you-sure" step, which will remove the SmartMirror Task definition from the system. The Duplicate this Task submit-button appears to re-display the same web page with all the same values (including any Task Name value), but is set to be handled as a New Task. This is intended to allow starting with a current SmartMirror Task definition that works, and is to be kept, but for which a variation is desired. For example, after a task is created to Push to Remote, Newer Files Only, and tested using Save Info and Run the Task Now, you can use Duplicate this Task and change the Direction to create a second task to Pull from Remote, Newer Files Only. The Local and Remote Settings are already set, the Task Name would have to be changed, and the Next Task to Run could be set to chain the two together. Encryption Select Encrypt Session to create a SmartMirror task that will use program ssh (Secure Shell) to encrypt the communication for the task. This choice may be prudent in some circumstances (e.g. company overseas office with intermediate routers controlled by an adversarial organization), required for others (e.g. if the data include personal health information or financial data), and illegal in some jurisdictions (generally police states). Everything that can be done with a non-Encrypted SmartMirror task can be done with an Encrypted task. There are a few differences --
In order for ssh-Secure-Shell to be used, a number of requirements must be met, some on the system running the SmartMirror task, and some on the Remote Server side of the operation. On the Remote Server: Public/Private keys must be generated, and the ssh-Secure-Shell protocol enabled. Check this on the Network -> Other web-page. On the Remote Server: The SmartMirror-Encrypted protocol must be enabled. (Note that the non-encrypted SmartMirror protocol may be either enabled or not, independent of SmartMirror-Encrypted.) Check this on the Network -> SmartMirror web-page. On the Remote Server: The Login Name to be used by the SmartMirror task must exist on the Remote Server, and access to the rsync-ssh operation must be allowed for that Login Name. On the Security -> Share-User-Permission, check that the Smirror/rsync-ssh operation is allowed for that Userid. The other special ssh operations, FTP-ssh and cmdline ssh, can be allowed or not, independent of rsync-ssh. On the Remote Server: For the Share which will be used in the SmartMirror task, ensure that the SmartMirror-Encrypted protocol is enabled for the Share. Because this is a new protocol, the default state for a previously existing share may be off, not allowed. Non-Encrypted SmartMirror protocol for the share can be enabled or not, independent of SmartMirror-Encrypted. Check this from the Storage -> Shares -> List Shares, selecting the details for the Share of interest. On the Remote Server: On that same Share-details admin web-page, ensure that the Userid is allowed to access the Share, either by explicit check-box or by being a Public share. This step is the same for either SmartMirror-Encrypted or non-encrypted SmartMirror. Of course, if that Share is read-only, then it cannot be the destination of the SmartMirror task. On the Remote Server, some of these steps can be combined in a single admin web-page check: Security -> Share-User-Permission will show (and can adjust) if the Userid has access to the Share and if the Userid is allowed to use Smirror/rsync-ssh. On the local host, where the SmartMirror task is being created or edited, one or more of the Public Keys of the Remote Server must be loaded, to mark those keys as "known" (and confirmed) in the ssh known_hosts file. During edit of a SmartMirror task with Encrypt Session selected, a check is made for Public Keys based on the Remote Address field, and if any are found, the SHA256 hash-fingerprint is displayed. If no Public Keys are found directly associated with the Remote Address, a link is displayed to Backup -> SSH Other Host Keys with encouragement to ensure that a suitable Public Key will be known. The ssh-Secure-Shell set of programs is very complicated and may be able to recognize and use a Public Key that the simple-minded Remote Address look-up does not recognize (such as a Public Key with an address field which is the ","-comma connected values of an IPv4 and IPv6 addresses with one or more dns.names). With all of these things in place, the Check Remote Host operation can be done, can list the Remote Server (non-hidden) Shares, and show the names of directories and files within the selected Share. The actual running of the Encrypted SmartMirror task, whether scheduled or chained, should be successful. What Could Possibly Go Wrong? For a SmartMirror-Encrypted operation, several things that can go wrong can be diagnosed using the Check Remote Host operation, the resulting error messages displayed on the web-page, and sometimes by checking the Status -> Login History on the Remote Server. Alas, there is no shortage of distraction. The message "Could not create directory '/home/.ssh'" provides no information, and is an artifact of the ZipZerver design. Also, the message in the Remote Host Login History (which may also appear on the Remote Server console) -- "Cannot get password from shadow file" -- also provides no information, and reflects the ZipZerver design to use the password file rather than a shadow-password file. Among the messages that might help --
SmartMirror Log FilesUpdated 2021-July-31 for set of log filesSmartMirror Log Files may be needed to help diagnose problems, and to provide output to give some assurance about what was done within a scheduled SmartMirror Task. SmartMirror Log Files can become much too large to be held within a ZerverModule flash memory area, and so are located on a Raid-Group. You can choose the RAID-Group or Share to hold Log File. Placing it within a Share will allow the Log Files to be visible (within the Share), and so can be examined by clients and users who have access to the Share. You can choose to place the Log Files within a Raid-Group in an area which is not accessible to any Share, which can be an advantage, sometimes. If so, the only access to the Log Files would be the links on this admin web page (and with the possibility of creating a Share definition for the root of the Raid-Group, which would make all files visible to that Share). Any non-empty FileName or Path/FileName for Log File can be chosen. SmartMirror is run so that any file with that name is excluded from rsync operations, to prevent reading or writing a file which is also being used as a Log File. A typical name might be "smirror.txt". The method is to use a set of files, each written up to a size boundary, with writing continuing on the successor file. Eventually the first file will be over-written. The files will be named, e.g. "smirror.txt" "smirror.txt.1" "smirror.txt.2" ... "smirror.txt.5" Under some conditions, there might still be a file named "smirror.txt.old" left over from the previous name scheme. The field Minimum Log File Size to Retain is in KBytes. Since there can be six files, one of which may have just been truncated for writing, the value (Log File Size /5) is used to set the limit for each file in the set. The previous log file method used only two files (named e.g. "smirror.txt.old" and "smirror.txt") and checked the file size limit only at the beginning of a SmartMirror task or chain of tasks. The result file could be much larger than the intended size limit, but did capture the log for the entire task or chain of tasks. The current method will not use much more disk space than the configured limit, perhaps no more than 20%. If the available disk space allows, there is no performance penalty for having a large limit value. The Log File Name and Size will be used that is in effect at the beginning of a SmartMirror task or chain of tasks. Use the submit-button Save Location and Name for Log File to change these values. The size is shown, in bytes, for the several parts of the log. The submit-button Discard older part of detail log file now will delete the oldest part, if any. If you repeat this operation, the remaining parts will be discarded, one by one. (If you delete all parts of a log file set while a SmartMirror task is running and still writing the most recent file, there should not be a problem -- by long-existing Unix/Linux design, the name will be removed from the directory, the file will still exist while open, and the delete which reclaims the space will occur when that file is closed.) The link Show SmartMirror detail log file will display the content of all pieces of the logfile. The link Show SmartMirror log file last 100Kbytes will be only the tail-end of what would be the full display. These are click-able links so that if your browser allows "right-click" and "Save Link Target as ..." (or similar), you may be able to save the result display as a text file. A long-accumulating log file could be 100 MB or more, and it sometimes happens that trying to open a browser window with that much text, even simple text, leads to problems.
A Log File entry begins with a time-stamp on a line with --
This is immediately followed by a line with the Task Name -- The bulk of the display is from the rsync program, including the command-line parameters and output as it runs showing progress, warnings, errors, and success.
If there are SmartMirror Tasks chained together,
each will be named with another line --
When a SmartMirror task has run which has no "Next Task" in a chain,
or if rsync ends with an error,
the log entry finishes with a time-stamp on the line --
SSH-Secure-Shell Known Hosts, Public KeysNew 2018-April-01The web page SSH-Secure-Shell Known Hosts, Public Keys is used to manage the local copy of the Public Keys of other systems, primarily for use with Encrypted SmartMirror tasks. The system is set up so that if a Public Key for a Remote Server is needed (for an Encrypted SmartMirror task) but none is known, the task will fail. (An alternative for a system with interactive users would try to acquire the Public Key from the (apparent) Remote system when needed, with an interactive dialog for confirmation that the right key was obtained.) Thus the Public Keys for use by Encrypted SmartMirror tasks must be acquired, and confirmed, in advance. If the ZipZerver has acquired any Public Keys, each will be listed with a checkbox that will allow selection for some of the operations described below. Each key is listed by the address field (IPv4, IPv6, or dns.name) and key-type. The full text of the key itself is not shown, but is represented by SHA256 hash-fingerprint value (256 bits represented by characters A-Z, a-z, 0-9, + and / at 6-bits per char) that would be used to confirm the proper key with each system that holds the Private part of a Public key. There is also a field for a Comment for each Public Key, if that will help keep things straight. There is nothing secret about each Public Key (thus the name), and the full text of the ssh GlobalKnownHostsFile can be seen or downloaded from the link SSH Other Host Keys as text file. The easiest way to acquire SSH Public Keys is with the two-step process --
The attack to counter here is called "Man In The Middle" (MITM), in which an adversary that can intercept the SmartMirror communication (perhaps by control of a router along the path to the Remote Server) can present a Public-Key that purports to be from the Remote Server but for which the adversary has the Private-part. The SmartMirror task uses encryption, but is communicating with the adversary, who decrypts everything and then re-encrypts to maintain a conversation with the actual Remote Server. Encryption is being used on both ends, but the Man In The Middle has a copy of the cleartext. The protection from using the wrong Public-Key is in step (2) above -- given the SHA256 fingerprint value of the Public-Key as presented, check that value as directly as possible with the fingerprint as known at the actual Remote Address. If the Scan... results would have crossed one or more routers controlled by a hostile organization and the SmartMirror task will be handling data that may affect The Fate Of The Free World, then talking directly to someone at the Remote Address whose voice you would recognize, comparing the full SHA256 value may be prudent. If the Remote Address is two cabinets over on the same LAN segment and the data seems mundane, maybe something a little less. Because there is almost no chance that an attacker has obtained control of a system in the next cabinet and can respond quicker than the actual intended Remote Address. Almost none. Any mismatch between the two SHA256 values may indicate a MITM attack, but is more likely to indicate that the Public Key in the list is out-of-date. The amount of attention here should depend on the nature of the security threat. If there are reasons that a Public-Key should not be used (perhaps including NSFW sequences in the random-like text within the SHA256 values?), select that key or keys, select Delete all selected keys now, and hit the submit button. The Edit Comment field... operation is implemented to allow adding a comment to one or more Public Key entries. The value can be any text that helps identify the Public Key, perhaps including the date that any SHA256 verify was done. The Edit the Address field... operation is implemented in anticipation of use with Certificate Authority keys, but the Mark selected key as a Certification Authority... and Mark ... as revoked... operations are mostly empty, ready for future work. The Load keys from file... operation is implemented and can be used with a file of one or more Public Keys, like the plain text file created by the link SSH Other Host Keys as text file. If ssh-Secure-Shell is running on a non-standard port (other than 22), for the system or systems listed in the Addresses field for a Scan... operation, then the Port field must be used for that number. Any such keys returned will be identified by the Remote-Address and the Port-number, since some systems might be able to run two different SSH services, with different Public/Private-Keys, on two different port numbers (ZipZerver does not do that). Ssh-Secure-Shell is a very complex set of programs, and there are lots of possibilities for key management, including Certificate-Authority keys that can verify the use of other keys within a range of addresses or dns.names. The minimal necessary set of operations for ssh Public Keys include -- Scan..., show fingerprint, and Delete. The other operations listed may allow future versions to be even more complicated and wonderous. With an even longer Help file.
System Status - Short or LongUpdated 2019-Nov-26 for Cert Expire displayThe System Information and Status admin web page has a short list of values which are important to understanding the system-state, including the Name and Location, identification of the firmware, and CPU Information (with one line for each CPU being used). To help describe the system state, this admin web-page will display all relevant nag-lines (rather than just one). There could be messages about reboot needed after firmware update, that the admin password has not been changed, that system startup is not complete, that the system has not been registered, and that no system-flash was found or recognized. If a System Message of the Day is defined, it will also be displayed. Beginning with firmware V1.1.14, both 32-bit and 64-bit versions are available. (Up to V1.1.08, only 32-bit versions were created. From V1.1.09 up to 1.1.13, only 64-bit versions were created.) The line HW lm-long-mode 64-bits will show whether the HardWare is 64-bit architecture. The line Firmware 32/64 mode will show which version of the firmware is currently running. These could be different, since 64-bit HW can run either 32-bit or 64-bit firmware. As a sanity-check on the Firmware 32/64 mode value, the field Memory Pointer Size shows the size of a memory pointer for the System Information and Status program (and presumably for all the other programs) as compiled and running. Also as a sanity-check between 32-bit and 64-bit versions, the field File Pointer Size shows the maximum possible file size, as understood by the compiled programs. A value of 32-bits would indicate a maximum file size of 4 GiB (actually, only 2 GiB since the 32-bits represent a signed value), which is a limit of some historical interest but is too small for many current applications. The expected value should be 64 bits, in which the real file-size limit is due to file-system implementation limitations (e.g. 16 TiB for all files held in a single ext3 file system), rather than a compiled-in smaller limit. The field Startup EFI bit size can also be 32 or 64, if the system boot-up process involved EFI-Enhanced-Firmware-Interface. The value can also be BIOS if that was the firmware involved with system startup. This value might be needed with some kinds of problems involving SW Update. Within the CPU Information can be fields showing how many bits are used to access memory, such as "34 bits physical, 32 bits virtual". The "32 bits virtual" indicates that a single program running on this hardware can access an address space of 4 GiB (2^32 addresses). If the info shows e.g. "48 bits virtual", with a value larger than 32, then a single program could access more than 4 GiB of memory provided that the Memory Pointer size is 64 bits. The "bits physical" value will show the memory size that can be addressed by the full set of programs (and which might or might not be larger than the "bits virtual" size). Independent of the theoretical memory limits for (single program) virtual addresses or (full system) physcial addresses, there is a field for Total Memory which is actually present and recognized in the memory slots by the ZipZerver firmware. The interpretation of the field Available Memory is not simple, in large part because Linux will tend to hang on to the buffers used to read and write particular files, after the I/O is complete, just in case the content of one of those files is needed again. So a large chunk of memory is not counted as Available, but it can, in fact, be re-allocated for another purpose immediately. A system which starts out busy with file operations will show the Available Memory drop steadily, lower and lower, seemingly headed for a memory crisis, but actually just retaining the buffers for more and more files in case that is useful. When the Available Memory drops below some limit, some of these potentially-useful buffers are picked and declared Available. As file operations continue, the Available Memory stays close to this lower limit. There is useful information in the Available Memory field, but the interpretation is not so simple. If the Time Monitor is running, the value for "stratum" will be displayed. Time servers at stratum=1 have direct access to a high-precision clock, such as a GPS receiver. A time server, such as ZipZerver, that synchronizes to a remote at stratum=N will then be running at stratum N + 1. Values from 1 to about 4 or 5 are good. Higher values indicate that there is quite a long chain of servers between the ZipZerver and a high-precision clock. A value of 16 indicates that no configured remote time servers are usable. This condition might be (briefly) reported immediately after the Time Monitor is started, but before an evaluation has finished. There are a couple of web-pages in which Certificates of different kinds might be obtained, and which can have time limits that can require administrator attention on a time-scale of a few months. The field Web https Cert Expire shows the expire date for an https-Web-Certificate, along with a link to the page for (many, many more) management choices. The field Reg-Support Cert Expire shows the expire-time for an installed Register-Support Certificate, if any, and a link to the relevant admin web-page. The field Periodic Disk Usage Report shows the time stamp for the most recent disk-usage check, if any. This does not require any administrator action, but is present as a reminder that this is a (possibly) useful tool that can be scheduled to run once a week. If there is not an expiring Certificate that needs attention, an administrator might choose instead to look at this report, to see if anything about the disk-usage patterns seems out of place. Also on this web page is a link marked System Status Internal Details which will show the details of the system state in great detail. This is intended partly for problem diagnosis and for development, and also for the curious. Among many items, for example, is a display for "meminfo" in which the state of Available Memory is described with about a dozen values. There may be system problems for which a copy of the contents of this web-page would be helpful. It is a selectable link so that if a web-browser allows "right-click" and "Save Link Target as ..." (or similar), then a text file can be easily created.
System Major Events Log (syslog)Updated 2023-Sept-05 (for change in date format) The events displayed by web page System Major Events Log (syslog) are written to ZerverModule Flash memory so that data is retained even after a system restart. (The Login History is also held in flash. The logs for "Recent Startup", "FileSys Op", and "Current Details" are held only in RAM, and can only have information from after the most recent system restart.) The Major Events include
These are events and conditions that may be of interest for some time -- days, weeks, or longer. Every hour or so, a system program runs to check the size of this file and if above a limit, trims it back. The submit-button Partial Discard from System Major Events Log File Now can be used to chop back the size of this file at any time, if there are older messages known to be of no interest, but with more recent messages that should be retained. The submit-button Clear System Major Events Log File Now can be used to clear all entries from this log. The other log files are unaffected. You can add your own entry to the Major Events Log File using the data field and the submit-button Add Text to Log File. This might be done to annotate a Log File to document an event not otherwise evident, such as "Reboot due to power failure". With version 1.2.11, available in 2023-Sept, the lower-level syslog tools have been updated and now use yyyy-mm-dd date format, noting the year as well as date of the year. Previous advice to add a "Happy New Year from 2023" message to document the year in long-lifetime log files is no longer needed. All these messages, and any annotations added, also are sent to the Current Detail log.
Login History and Connect AttemptsUpdated 2017-Dec-16 The Login History and Connect Attempts admin web page is provided to answer questions about who connected to what from where when. The intention is to have each protocol put relevant login, logout, login-failure, and login-deny events here. Information is recorded for sessions from Windows-SMB, FTP-File-Transfer-Protocol, and rsync-Smart-Mirror. Unix-NFS protocol does not use sessions that can be associated with a userid, and so is not recorded in Login History. HTTP does not really use sessions, and rather than have every access create a Login History entry, HTTP access is noted in Current Details Log and in WebAccess Log. This login log is kept in ZerverModule flash memory, so as to be available over a system restart, similar to the Major Events Log. (The log files "Recent Startup", "FileSys Op", and "Current Details" are kept only in RAM, and so only have information from after the most recent system restart.) The maximum size of the Login History is limited, so that a series of connections or connection-attempts cannot cause the system to run out of space in ZerverModule flash memory. This means that the Login History may contain information covering weeks, days, hours, or perhaps only a few minutes. The method involves using several files (e.g. 5) in a file-set, and when each file grows to a limit-size, then the next file in the set is opened to continue writing. If all files are in use, this has the effect of automatically discarding the oldest information. Previous versions of this system used a single file for Login History, with a program that would run (about once an hour) to trim back the size above a limit. During an hour, that file could grow (perhaps under an attack of external login attempts) to exhaust all space in ZerverModule flash, with various result effects, generally bad. An odd symptom will occur if this system is run with this firmware writing Login History to the files of a file-set, and then the system is re-flashed back to earlier firmware which expects only a single file. That system would show only the part of Login History which happened to be written to the first file of the file-set. New Login History entries would be written and displayed by that older firmware, but a later re-upgrade to newer firmware may show clumps of entries in a somewhat scrambled way. The Samba program running the Windows SMB/CIFS protocol has, as a feature, the ability to note Begin-Use and End-Use of a Share. This can occur after a login, as Shares are mapped and perhaps unmapped. If the Share or Shares are Public, there might not be an SMB login recorded at all. The Samba Begin-Use End-Use messages can be a problem with some applications or system services, perhaps because of using UNC access rather than a mapped drive, which can Begin-Use and End-Use several times a minute, filling the Login History file and obliterating other useful events. The admin web page Network -> Windows has a control to stop the Begin-use, End-use messages. As with the Major Events Log, for systems that are configured to Listen for Remote Syslog events From Other Systems (as configured on the System -> General admin web page), the events from other systems configured to send Login History will be recorded. Also as with the Major Events Log, there is a submit-button for Partial Discard from Login History File Now, which is done by stepping to the next file in the file-set, and which will appear to have had no effect if the Login History informamtion does not yet use all the files in the file-set. Annotations can be made using the data-entry field and the Add Text to Log File submit-button. Entries to the Login History are also added to the Current Details log.
Most Recent System Startup Details LogThe admin web page for the Most Recent System Startup Details Log is intended to be all available details, showing hardware discovery and configuration steps.Before the ZerverModule has been located and configuration information found, the ZipZerver does not know the right Network Name to use, nor what TZ Time Zone is desired for the time-stamp values. Initially, the Startup Log uses the name zGMT and shows time-stamp values in UTC - Universal Co-ordinated Time, formerly known as GMT - Greenwich Mean Time. There can be a large number of entries from the beginning of start-up with the same time-stamp value. This is misleading. A large number of kernel start-up messages will be buffered, and eventually the syslog facility (and other programs) will be ready to receive those messages. The initial burst of messages has the time-stamp of when these messages were received by the syslog facility, not the time-stamp of when they were generated. Eventually the time-stamp values will show a gap of a few seconds or more. At that point, the initial kernel message burst has been processed, and subsequent messages in the syslog would have time-stamps, or perhaps time-stamp differences, that are more meaningful. The initial message in this log is likely to be tagged with syslogd restart, to denote that at some point during start-up, syslog began, and then received the kernel start-up messages. The kernel messages should show that CPUs are counted, hardware is discovered, software modules are loaded, hardware is initialized. Eventually ZERVER Mount config device ... as /flash occurs. Shortly after this, the configured (or default) name is found, the desired (or default) Time Zone is found, and the Startup Log messages begin to make use of both. After this, network interfaces are enabled, protocols started, Raid-Groups assembled, Shares and access are configured. Some network protocols and startup-steps take quite a few seconds to become fully operational, and the last message about success or failure for those might occur after the arbitrary point at which the Startup phase is declared to be done. Any such messages go to the Current Details log. One possible difficulty -- this file actually begins as the Current Details log, which is renamed to make the Startup Log. For a system with a lot of hardware, such as 16 disk drives, in which there were a lot of messages about each disk, the size-limit for the Current Details log could be hit, and older text would be discarded. The beginning of the Startup Log file would have lost some of the beginning events.
Most Recent File System Operation Details LogThe web page for Most Recent File System Operation Details Log is likely to have a very few lines generated during the most recent system start-up, showing a File-System Check was done on /dev/md0 and for the name of the File-System will announce "clean", with the number of files and directories as a fraction of the maximum number, and the number of blocks used as a fraction of the maximum number.If the previous shutdown was not entirely clean, there may be a line about recovering the journal. If applying the journal does not bring the File-System to a clean state, or if a File-System Check operation was forced from the Storage -> List RAID Groups web page, then there will be a longer display. If problems were found and fixed, or (worse) were found and could not be fixed, the details will be found with this web page. The handling of a File-System with serious damage is beyond the scope of these notes -- call Customer Support. The real motivation for a web page to display the results of a complex and perhaps time-consuming operation comes from Raid-Group Create. There are a number of steps, and when Raid-Group Creation has failed, it has been because one or more of the disks intended as component pieces were in some un-anticipated state, such as low-level formatted for a block-size different from 512-bytes. Not often, but every once in a while, nothing will do but to look at the result log. Although there are a considerable number of complex steps, the size of the log file for Raid-Group Create will not be too large. A File-System Check, on the other hand, that encountered serious problems could create quite a lot of output. So as not to run out of space in the region of RAM, the File-System Check is run so as to keep the first 60 lines and the last 60 lines of output. If there are minor problems, this should be enough to describe what was found and what was done to repair. If output is discarded, the line-count will be shown. Whether a handful or hundreds, the discarded lines mean that the file-system is a mess, and back-up copies may need to be used. The submit-button Clear FileSys Operation Log Now is used to discard this file from RAM.
System Operation - Current DetailsUpdated 2017-Dec-16 The Current Details From Most Recent System Restart admin web page records the maximum available detail for all system events. The Current Details log file is kept in RAM, and so contains only events from after the most recent system restart, starting at the point where the log Recent Startup ends. This is in contrast to log files Major Events Log and Login History which are written to ZerverModule flash memory to record events over multiple system restarts. The Current Details log file is size-limited, so that no matter how rapidly some run-away program generates event messages, all events can be sent to the log without danger of running the memory area out of space and complicating diagnosis of the real problem. The size-limit means that the log may contain messages from several days, several hours, or perhaps only covering a few minutes. Copies of events sent to the Major Events Log and Login History also appear in the Current Details, but will be pushed off the end by more recent events much more quickly. For systems that are configured to Listen for Remote Syslog events From Other Systems (as configured on the System -> General admin web page), the events from other systems configured to send All Events will be recorded. As with Major Events Log and Login History, annotations can be added to the Current Details log using the data field and the Add Text to Log File submit-button, perhaps useful before sending a copy of a log to Customer Support. A Partial Discard from Current Details Log Now submit-button can be used to discard some detail messages considered irrelevant. This affects the Current Details log only -- the other log files, especially Major Events Log, are unchanged. As with the Login History Log, the method for keeping the most recent information involves writing to one file of a file-set until a file-size-limit is reached, and then stepping to the next file in the file-set. The Partial Discard will appear to have no effect if the Current Details Log does not yet use all the files in the file-set.
Web Access Requests and ResponsesUpdated 2023-Sept-05 (adjust note about Time field) The log of recent HTTP web server requests is kept in RAM, and so includes only events since the most recent startup. Since it is size-limited to not allow RAM to be filled, there may be entries from multiple days, hours, or perhaps only the most recent few minutes. The Time field appears to respond to changes in the system's time, but not to changes in the system's time-zone since startup. This value is when the request was received, but appears in the log in the order that requests are finished (and the result size is known). Time-stamps can appear (slightly) out or order, and the GET for a small image file can appear before the GET of the web-page with the reference to that image file. The Host field will be the IP address of the client. The Remote User field may identify the userid used from the client, may be bogus if the final status (see below) is 401-unauthorized, or may be "-" empty. The Request entry is the first line of the request. The Final Status field is a 3-digit number with "200" indicating success, and other values for various degrees of success. The Size is the number of bytes in the response, with "-" used for no-response, rather than "0". The Referer field may show the web-page from which this request has come.
Answers to I-Am-Not-A-Robot selectionNew on 2021-Aug-10The page allows a few config choices for handling of the 'I am not a robot' selection (initially used for a 'Contact Us' webpage that would submit an email to the system administrator), show counts for answers selected, and show any user-supplied answers. This webpage is the result of annoyance over an 'I am not a robot' interface in a frequently-encountered application. For the Show Robot Question, Select No to avoid all of this stuff. If the Robot question is to be shown, an answer can either be Required or be left as Optional. If the operation, such as a 'Contact Us' webpage, is being abused, Supress entire operation which uses robot question can be set. If the allow user text entry field is Yes, then the Robot question will have, as a possible answer, a text entry line. If the show entries from other users field is Yes, any such accumulated answers will be displayed (up to a limit), and those will be among the selectable answers. If No, then only the predefined answers and administrator curated answers (discussed below) will be available as possible answers. Because the edit of the set of curated answers may take multiple steps, these config values are not stored until the Save results button is hit. The Curr Config values are shown separately from the config choices, since they can differ until the edit and adjustments are saved. The set of predefined answers, the set of administrator curated answers, and the set of user-supplied answers occurs next, along with an edit box, for text entry or adjustments. The set of predefined answers, including 'I am not a robot', cannot be deleted or edited, and so cannot be selected. Slots for administrator-curated answers are marked from (0) to (9). When one is selected, any text in the edit box will be moved to that slot by hitting the Exchange or Copy selected value submit button. The text is limited in length to about 120 characters, and (like the user-supplied values) an attempt has been made to discourage text which uses any of the Seven Words You Cannot Say on TV. Any user-supplied answers will appear below slot (9), and one can be selected and moved to the edit box, punctuation added, and then moved to a curated slot by using the Exchange or Copy button. The curated slots content can be re-ordered by Exchanges with the edit box. There are limited slots for curated answers, and limited space for user-supplied answers. Items that have outlived their entertainment value can be removed with the Delete selected value submit button. A series of changes to the available answers or change to the config values is completed by hitting the Save results of any edits/adjustments submit button. Because several steps might be taken in adjusting a set of available answers, those changes are held in an ephemeral way (as webpage parameter values). If an editing session is going poorly, one could abandon a series of changes by hitting the Abandon any edits and adjustments submit button. This is equivalent to re-enter of this webpage from a navigate link. The apparent purpose of all this mechanism is to show counts of which answers have been selected or entered. The curated answers are kept in flash, but the user-supplied answers and the counts for all entries are kept in RAM, and thus will be lost after a reboot. An answer that is edited in a way that changes spelling might not be recognized as the same string, and a count value might be lost. And the counts are not maintained by atomic transactions -- near-concurrent changes are not guaranteed to maintain values. Use these numbers, if any, for entertainment purposes only.
Agreement and LicensesUpdated 2023-Sept-05 (to drop ref to "friendly customer support") The text of the License agreement speaks for itself. Any attempt to elaborate on the meaning, if any, or try to explain what parts of a ZipZerver are covered, and why, would just detract from the simple elegance of its timeless phrases. Or something. The information on the Agreement and Licenses admin web page involves the components of the product. A Slackware 10.1 distribution was the original starting point (update as of 2022-Sept uses Slackware-15.0), in part because the start-up scripts were simple enough to still be within possible human understanding. Various component pieces have been compiled from source code, to support the desired configuration and security choices. For each component compiled from source code, the License which allows us to use it is included, most often "GPLv2", the "GNU Public License, Version 2". At this writing, GPLv3 has been created, and might be used for future versions for one, or more, or all of these components. The version number of each of these components is shown for reference. Q: For FOSS Free and Open-Source Software, shouldn't the price always be free? The preamble to GPLv2 has a good discussion of this question. Since this is a web page description, here is a bumper-sticker-sized summary as frequently seen on website slashdot.org -- "Free as in Free Speech, not Free Beer." Q: Can I download each of the pieces using the links from the Agreement page, do a few compiles, and make my own system ? Yes. Q: How would such a quick home-grown system differ from a ZipZerver ? It would not be as pleasant to install, boot-up, configure disks and Raid-Groups, configure shares, allocate permissions, monitor, maintain, backup with rsync, and update, including thorough testing, without considerably more work. Configuration would involve careful editing of a number of interacting config files. Perhaps most significantly, there would not be thoughtful, carefully researched, generally informative, occasionally amusing little essays available on each web page with a link marked "Help".
Register and SupportUpdated 2019-May-12In order to encourage a ZipZerver to be registered for on-going support, there is a mechanism that notes the configured sizes of all Raid-Groups and when the current agreement for Support will expire. If a Certificate has been installed, the Register and Support admin web page will show the values.
The values for Current Configured Raid-Group Status --
Nothing happens right away. The point of a ZipZerver is to keep your data safe and accessible, and that remains a fundamental consideration even if a Certificate is expired. The following steps are each likely to be a day or more --
The multi-day response time is, in part, so that providing an updated Certificate is not a failure-point in a reboot nor a road-block in a recovery situation. Some organizations are big enough that, 24-hours a day, someone is available to deal with an account. The ZipZerver organization is not one of them. When you renew a Certificate, there may be a day or two of turnaround time, and perhaps longer during 3-day weekends. To ensure that your data is safe and available, that last step, Un-mount each Raid-Group, will not be done for two days (or more) after a reboot. In a worst-case situation, in which an updated Certificate can not be obtained for any reason, your data will still be accessible and usable, with the aggravation of having to reboot the ZipZerver every two days. A particular situation -- after hardware catastrophe, if a Configuration-Save file has to be used there should not be a roadblock to recovery of access to the data, regardless of whether the Config-Restore operation has recovered a now-Expired Certificate or a non-Expired Certificate is now running on unexpected hardware. Auto Certificate Check can be set, depending on a suitable value for Certificate Next Check URL, a choice made for Certificate Next Check HW ID, selecting to ... enable periodic Cert Check ... and hitting the Start Certificate Operation Now submit button. The periodic check for an updated Certificate will be done about once a week, unless the Certificate Status is Near-Expire, and then will be tried every hour. If unexpected things occur (or expected things do not occur), clues might be obtained from admin web page Status -> Current Details. Under some conditions, the full log of an attempted Cert Check operation might be found at admin web page Status -> FileSys Op.
Unexpected fnc Parameter ValueA message of the form aweb_get_input result = -2 can occur if the web server that should handle web requests somehow mangles the parameters, or if a program which is expecting an artfully constructed set of web parameters prepared by the web server is actually run in some different environment, such as from a command line. An error message of the form Unexpected value for fnc can occur if the set of admin web pages are generated by a set of programs that are not fully co-ordinated. One program has created a web page with a link or a form to invoke another program (or even the same program), and that target program finds that parameter fnc has an un-expected value. One program (or piece of code) is a bit ahead of the other, in development.
Change a Passphrase for a UseridThe Change a Passphrase for a Userid web page allows a user to change the passphrase that secures the userid.
AboutThe "About" web page provides information about the ZipZerver product, how to contact the organization for support, and the version of the firmware that is running. Defer Delete for SMB/CIFS ProtocolTo help with a performance limitation of a Windows system using a single connection to a ZipZerver, a feature is available so that file-delete operations can be done in the background. A Windows system can be running multiple applications, or running an application with multiple threads or multiple processes which together are manipulating dozens or hundreds of files at a time. However, the Windows system only establishes a single SMB/CIFS protocol connection to the ZipZerver, and all of these file manipulations are multiplexed over this single TCP/IP connection. There is a considerable amount of pipe-lining and overlap for these multiple operations on multiple files, and as long as each individual SMB/CIFS request is small, then the operation works well. However, the process-time required for a file-delete operation is not related to the size of the request -- it is related to the size of the file. A delete of a file involves returning every block back to the free pool. A four gigabyte file has a million blocks, and the delete operation can take many seconds. During this time, processing of the queue of file requests from that one Windows System over the one SMB/CIFS TCP/IP connection comes to a halt until the delete is completed. This can create performance or responsiveness problems for the applications running on the Windows system. (Other protocols which could handle multiple client processes making concurrent requests to a single ZipZerver use one connection for each client process, rather than multiplex all requests over a single connection from the node. Delete of a multi-gigabyte file still takes time, but only one client process is held back waiting for the response.) To try to avoid this performance problem, file delete operations done with SMB/CIFS protocol to a Samba task can be done in the background -- if the directory for a share contains a directory named ".defer_delete" when the share is defined or when share-attributes are being evaluated, then a file-delete operation is done by a rename of the file so that it appears within the .defer_delete directory. The rename operation does not depend on the size of the file, and so the response to the delete request occurs quickly. A task named "deferdeleter" runs in the background, and every few minutes will check each share for the presence of a .defer_delete directory with files to be deleted. Sometime shortly after midnight, this task puts a report in the detailed event log describing the number of files found and deleted, and summarizing any problems found. Because of the leading "."-dot character in the name ".defer_delete", several protocols treat this directory as having the "hidden" attribute. This may complicate checking for its presence. Cleaning up a file-system that is low on space may present some confusing symptoms, if Windows and SMB/CIFS protocol is used to delete unneeded files. Gigabytes of files can be selected and deleted, but a check of Raid-Group or file-system information may show that the available space has not changed at all. This would occur if those files have simply been moved into the .defer_delete directory, and are not yet gone. A check ten minutes later will show that the space is now available.
Board Set-up and the Security Back-DoorFor BIOS and EFI set-up for a board to be used with a ZerverModule --
Note quirky spelling of the command name. When this command is run as the root super-user, the previous value of the password for admin is not required, and a new value can be set. This will allow login to the admin web pages, where the General web-page should be used to set the admin password again, so that various inter-related password values are all properly set. For the cable that would be needed -- for the PC-board with a ZerverModule, under other conditions, the first serial-port could be used with a modem. For a PC running a program to emulate a 38400 baud terminal, the serial port also could be used with a modem. Therefore the cable and adapters that would connect these two serial ports, whether 9-pin, 25-pin, or something else, should look like a "Null modem".
Bootup Firmware BIOS, EFI, UEFINew 2021-June-23Prior to release 1.2.06, ZipZerver was expected to boot-up on PC-class hardware using the BIOS-Basic-Input-Output-System, the time-tested (several decades) mechanism in which bootable devices could be selected and ordered (including, once upon a time, floppy disks), each tried in order to load a small amount of code, and if successful, to begin executing that code to begin the many steps of loading a full operating system. A newer and much more complex system, EFI-Extensible-Firmware-Interface and UEFI-Unified-EFI, has been available for a few years. This has not been a problem as long as any hardware running the UEFI code had a Legacy or Compatability Mode (or other obscure name) which would still handle BIOS-loaded operating systems. The inevitable finally did occur. PC-class hardware was encountered which was EFI and which did not appear to have a Legacy or Compatability mode. Boot-up under EFI was going to be handled. How It Works (or is supposed to) In the following discussion, the name refind.efi is used to refer to the file for the boot-manager program refind. The actual file name is either refind_x64.efi or refind_ia32.efi depending on whether 64-bit or 32-bit HW is being used.
Why EFI The boot-up process can be tricky, the programming environment requires maximum privilege to discover and initialize hardware, and BIOS code had to run in the most primitive addressing environment. When things went wrong the system would hang, there was usually no hint as to the problem, and the only choice was between reset and power-off. EFI has many differences with BIOS.
A web-site that was very helpful in getting EFI to work is Managing EFI Boot Loaders for Linux by Rod Smith. He has adapted an early project called rEFIt under the name rEFInd which appeared to be the best choice for a boot manager for ZipZerver, and provided extensive discussion of principles and practical choices (and is an actual writer). If you need a deeper understanding of EFI and the things that can go wrong, this web page is worth your time (and well worth a $10 Donate). Expected EFI Choices from a Bootable-CD For boot-up using a ZipZerver bootable-CD (or CD image file in a context such as VirtualBox), EFI boot-manager program refind.efi should run and present several choices --
Expected EFI Choices from a flash config-device For EFI boot-up directly from a config-device (or after bootable-CD in which the "EFI\refind\refind_x64.efi from EFIBOOT" line can be chosen), the refind.efi boot-manager choices include one or more of --
If EFI Boot-up Shows None of Those Choices This section will try to describe steps that can be taken to achieve a successful boot, repeatable even. The next section contains a dump of details and aggravations that might (or might not) help in understanding these steps.
EFI seems to consult a set of BootXXXX
NVRAM-Non-Volatile-RAM variables
associated with the cpu board
to decide what bootable devices
and what sub-directories of the EFI directory should be run.
To boot from a flash config-device,
an entry must exist for --
To boot only once from a CD (such as for an install),
there may be simpler steps given below.
To repeatably boot-up from a CD,
those same file names are in a different FAT FS, and
an entry must exist for --
The BootOrder must be adjusted so that refind.efi is first.
Alternatively (untested),
any existing entries can be deleted or marked as inactive.
The theory is that in the absence of any active BootXXXX values,
the default path will be used,
which exists on both bootable-CD and flash config-device
and are copies of refind.efi --
In testing, if the refind.efi choices were not presented, several things could occur.
How Complicated Is It? The EFI facility can have a lot of choices. Of the devices visible at bootup, several might be recognized as being EFI bootable, such as both a bootable-CD and a flash config-device. There may be firmware choices to set the order (similar to BIOS). Within the FAT FS of such a bootable device, there can be multiple os-loaders and/or multiple boot-managers, each located in a different directory under the EFI directory. These will be tried in some order until something works. For ZipZerver, the EFI directory, in both a bootable-CD and a config-device, will have only two subdirectories -- one named refind and one named boot that is supposed to be a fallback default if no other directory works. Both of these will run refind.efi and have identical refind.conf files. If this is a system with disks left over from other uses, it is possible for EFI to discover one or more additional FAT FS that look like ESP-EFI-System-Partitions that could have multiple directories for boot-managers and boot-loaders. Some EFI appear to be limited to just FAT FS at this stage, and others (such as VirtualBox) appear able to read other FS, such as the iso9660 FS of a CD. When a boot-manager runs, such as refind.efi (or GRUB2), there can be more choices. refind.efi can have drivers for additional FS types, and in ZipZerver there are drivers for Linux ext2 and ext4 FS along with iso9660. This allows files to be loaded from where the SW Update operation puts them (rather than from within the FAT FS). Once refind.efi is run, its .conf will present some choices in some order (such as Zerver_Next and Zerver_Previous) with a default choice and a timeout. One of those choices is to look high and low for other .efi or bootable operating system choices and present them as part of the list. And of course one of the things that refind.efi can find is another boot-manager such as in the bootable-CD list in which the refind.efi from a flash config-device can be found. (The .conf for that later instance is not configured to look at external devices, so it does not find a choice to go back to the bootable-CD.) Disks that are not yet integrated into a ZipZerver Raid-Group might have left-over OS or EFI-boot-managers (such as GRUB2) presented as choices. There are some opportunities here for confusion. At some level, it seems as if EFI should be looking for devices with ESP-EFI-System-Partitions, trying to run .efi programs within each directory within the EFI directory, and if that fails (that is, control returns to EFI rather than going to an operating system), to try another. When a boot-manager is run as part of this process, such as refind.efi, choices are presented and a default chosen after a timeout period. If that is how it actually works, then it seems like some operating system should be loaded and run. This is where the NVRAM-Non-Volatile-RAM-Random-Access-Memory of the board is used in a way that is independent of any one operating system. Along with picking a boot-device, it appears that a device-partition, directory-path-name, and parameters can be set to be used by the selected operating system. These variables have names such as Boot0000 (up to Boot0006 for an example at hand) that can be visible and adjustable within each operating system and by an elaborate Hardware Setup Utility that is part of EFI. During development of the use of EFI for ZipZerver, these NVRAM Boot-values did not appear to help very much. If the cpu board to be used as a ZipZerver is being re-purposed, and the previous system disk is still present, there is a good chance that one of those BootXXXX variables will call for loading the previous operating system. The entries may be non-volatile, but an entry may stop working at some point. One way that can occur -- disks are identified by a 128-bit UUID, expected to be unique (rather than as "the first disk", which can change with slight changes in reboot timing). If something changes the UUID of a flash config-device (as can happen in a development environment), then the path and file-name remain, but the disk does not appear to be present. Notes about EFI that should go somewhere These notes apply to "PC-class" cpu boards that would be capable of running Windows, and depend on recognizing a FAT File-System. However, Macintosh boards also use EFI (in some form), but their File-System is HFS-Hierarchical-File-System (it is a Macintosh thing) rather than FAT FS. Bootable-CD and config-device for ZipZerver would not be recognized by such Macintosh boards. Why use boot-manager program refind.efi?
Useful Command-line Operations for adminThe intention is that all system operations can be handled using the admin web pages. However, there could be circumstances that were not anticipated, are not handled by the admin web page operations, that can be handled by the admin user with a command-line operation. A video console is not required on a ZipZerver. If one is present, the system start-up messages, along with any other critial system error messages, are likely to appear there. (Those start-up and critical messages can be configured to go to a terminal on the first serial port, and so might not appear on a connected video monitor.) A "login:" prompt does not appear automatically. In order to get a "login:" prompt from the system console, you must use key-combination Alt-F2 to select an alternate virtual terminal after system start-up is complete.
Reset a Flash to Factory Default StateIf a ZerverModule or ZipZerver flash key must be reset to factory default condition, this can be done by command from a keyboard and console admin login. Such an operation would not affect any Raid-Group, either active or off-line, attached to the ZipZerver, nor affect any of the files on the Raid-Group. However the Share definitions will be deleted (along with all other configuration choices), so those files will no longer be accessible or visible. In order to get a "login:" prompt from the system console, you must use key-combination Alt-F2 to select an alternate virtual terminal after system start-up is complete. Login as admin.
The command below has an odd name, including text "...init_yes",
because there may not be any "...are you sure?" chances
for second thoughts --
There will be a check to see if there are any Raid-Groups currently mounted, and if found, there will be an interactive chance to abandon the factory_init command before any changes are made. Otherwise, all configured data, IP addresses, userid, share definitions, passwords, and any other ZipZerver configuration will be reset. At the next reboot using the ZerverModule on the system or on any other system, everything will be in the factory-default state. The last steps in the script provides a chance to either shutdown or reboot the system.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
HTTPS=on SSL_TLS_SNI=polycosmic-zervertest.zapto.org HTTP_ACCEPT=*/* HTTP_USER_AGENT=Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com) HTTP_ACCEPT_ENCODING=gzip, br, zstd, deflate HTTP_HOST=polycosmic-zervertest.zapto.org PATH=/sbin:/usr/sbin:/bin:/usr/bin SERVER_SIGNATURE= SERVER_SOFTWARE=Apache/2.4.62 (Unix) OpenSSL/3.4.0 SERVER_NAME=polycosmic-zervertest.zapto.org SERVER_ADDR=108.181.171.214 SERVER_PORT=443 REMOTE_ADDR=3.144.7.216 DOCUMENT_ROOT=/home/shares REQUEST_SCHEME=https CONTEXT_PREFIX= CONTEXT_DOCUMENT_ROOT=/home/shares SERVER_ADMIN=you@example.com SCRIPT_FILENAME=/home/shares/web/v1/fullhelp.html REMOTE_PORT=59614 GATEWAY_INTERFACE=CGI/1.1 SERVER_PROTOCOL=HTTP/1.1 REQUEST_METHOD=GET QUERY_STRING= REQUEST_URI=/web/v1/fullhelp.html SCRIPT_NAME=/web/v1/fullhelp.html DATE_LOCAL=Sunday, 15-Dec-2024 03:54:15 MST DATE_GMT=Sunday, 15-Dec-2024 10:54:15 GMT LAST_MODIFIED=Tuesday, 03-Dec-2024 21:12:47 MST DOCUMENT_URI=/web/v1/fullhelp.html DOCUMENT_ARGS= USER_NAME=public DOCUMENT_NAME=fullhelp.html