Blog: World Backup Day 2015 - Time for a Backup Checkup
Mar 27, 2015
Blog: New Google Calendar App for iOS devices
Mar 16, 2015
Blog: Planned upgrade of IDM equipment - Friday March 13, 4pm
Mar 12, 2015
Blog: Unscheduled outage - Google Apps Marketplace Apps - March 5
Mar 05, 2015
Blog: Smartsheet Unscheduled Outage - March 4 2015 - Resolved
Mar 04, 2015
Blog: Intel® Xeon Phi™ Coprocessor Developer Training
Feb 28, 2015
Blog: IT and ESNet Help ALS Researchers Move their Big Data
Feb 28, 2015
Blog: Unscheduled IT Outage Feb 27 2015
Feb 27, 2015
Blog: New Password Reset Service Entering "Registration" Phase
Feb 23, 2015
Blog: 5+ Incredible IT Training Opportunities Available Now!
Feb 13, 2015
Blog: WWW Today Newscenter and Wordpress Hosted Sites Scheduled Outage Feb 2 2015 8PM
Jan 26, 2015
Blog: Archiving Files/Records Class - January 29
Jan 07, 2015
World Backup Day: Is your data backed up?
The IT Division is taking part in celebrating World Backup Day by encouraging the Lab community to double check that your backups are working and ensure that all your important data is backed up.
Don't Be An April Fool - Do A Backup Checkup
It's time for your checkup. We promise it won't hurt.
First: Check up on your strategy. What are you trying to achieve with your backup? Are you trying to backup all the data in your experiments, or just some, or just your important findings? Do your backups need to be able to survive a major earthquake that impacted the site? Are you using the best form of technology to ensure that your data remains safe? For example, if you're still using external hard drives as backup, make sure you've evaluated some of the newer alternatives that may provide more resilient backups. Remember that file sync (like Dropbox and Google Drive Sync) are not the same thing as Backups (see more on this below).
Second: Check up on your scope. Are you actually backing up the data you want to backup? Have your experimental results moved somewhere else and you're no longer backing them up? Make sure your backup software or process is correctly backing up the files and directories and systems you need.
Third: Check up on your data. Now it's time to go do a quick spot check on your backups. Does your backup client report that it's working? Can you see recent files in your backups or in the logs provided by the backup client? Does the size of your backup correspond to the amount of data you think you've backed up?
What if I don't know the answers?
If you can't complete the backup checkup because you're using systems managed by others, now is the time to ask some questions! Find your sysadmin or another cognizant person and confirm what and how is being backed up on your behalf. Ask them to run through the backup checkup too.
Can IT Help Me Backup My Data?
IT offers various options to help from simple desktop/laptop backup solutions (Carbonite) to infrastructure like Google Drive that is already backed up, to more complex backups for servers and shared storage.
What Else Should I Know?
A Quick Word About External Hard Drives
Historically, nothing has competed with external hard drives for ease and cost of doing major backups for research data. However, that's starting to change. Cloud services like Google Drive and Carbonite provide reasonably speedy and large volume alternatives at competitive prices (or even free). While External Hard Drives are pretty good, they do have surprising failure rates and, unless they are reliably stored offsite, they are unlikely to allow your research data to survive a major event at the Lab (or even a minor one like a particularly nasty virus or a fire sprinkler release). If you use external hard drives, take a minute to consider other options. Need help, contact email@example.com
A Quick Word About Google Drive Sync and Dropbox and Other File Synchronization Services
File synchronization services like the Google Drive Sync Client and Dropbox provide some of the features of backups but are not, fundamentally, backup tools. This is because sync clients are highly susceptible to accidental local changes that propagate through the backup This is even more true in collaborative file sharing environments where it's possible that a collaborator could accidentally delete your important folder or file and, potentially, delete your local copy as well! While file synchronization does provide some resiliency, it doesn't equate to a full backup solution.
However, you can safely make use of the Lab's Google Drive storage space as a backup location (all employees have 30GB or shared mail and drive space) by doing the following:
- Create a folder for your backup in the web interface of Google Drive (not the file browser on your computer). Make sure it's named something obvious and don't share it with anyone.
- Ensure that all your local Google Drive Sync clients are set to choose the directories you want to sync and make sure that new backup file is excluded.
- Now use the web interface or file uploader interface at drive.lbl.gov to upload files.
Provided you don't accidentally delete the files or accidentally begin syncing these files locally, this should provide a safe backup destination for your work. Need help, contact firstname.lastname@example.org
Following the successful launch of a Calendar app for Android last Fall, Google released a version for iOS devices last week.
For "power" calendar users, we hope this solves some of the issues we have seen with the built-in calendar app on iPhones. The description for the Android app is here.
Reference the Google Blog for a brief intro and a link to Apple's AppStore.
After downloading the app, tap the Google Calendar icon and watch the intro slides. You will then be prompted to login at the Google login screen, (only enter your full LBL email address at this stage, leave the password field blank and continue). This will redirect you to LBL's Single Sign-On page where you will enter your Berkeley Lab Identity credentials.
Once you are in the app, look at the top left hand corner and you will see a 3 bar icon, tap the icon and you will see the options available, such as what calendars to display. Settings are found at the bottom of this list: tap settings to add multiple Google accounts by tapping "Manage Accounts"
The Identity Management Team will perform a planned upgrade to equipment that front ends our services (the enterprise directory, authentication systems, phonebook, etc). This is a security upgrade and will be done at 4pm, Friday March 13. The outage window is 30 minutes or less. Anyone already logged into a lab system will not be impacted.
From approximately 7:30am through 9:00am March 5, a problem at Google resulted in authentication failures when Lab users attempted to use any of our Marketplace Apps (Lucidchart, Smartsheet, GQueues . etc).
This issue is resolved.
Smartsheet is currently experiencing an issue which prevents users from logging in. More at:
Intel is sponsoring a free one-day in-depth training on the Xeon Phi Coprocessor to be held at Perseverance Hall on March 27th, 2015 from 9:00am to 4:00pm. This training will provide software developers the foundation needed for modernizing their code to take advantage of parallel architectures found in both the Intel® Xeon® processor and the Intel® Xeon Phi™ coprocessor, which are currently available to Lab researchers and collaborators on the LBNL Lawrencium Cluster.
Lunch will be provided. For more information and registration, please go here
. This event, hosted by the IT Division High Performance Computing Services Group
, is open to all LBNL and UC Berkeley staff and faculty. Space is limited so please register early!
Working at the ALS generates huge amounts of data, and for many years this has caused users to have to carry hard drives and USB drives between the ALS and their home institutions for acquisition and analysis of experimental data. To avoid the physical transport of data and to make real-time analysis possible, staff at the ALS, ESnet, and Berkeley Lab's IT Division have collaborated to implement several best practices that allow the fast and secure transfer of data over the network to a users home institution. A case study, performed by ESNet, highlights the work of IT Division staff, Susan James, Yong Qin, and Karen Fernsler to build the Data Transfer Node and 10GBE network, integrate it with the data acquisition system and implement the Globus Online data transfer tools. The end result shows the improved workflow and data export for the x-ray tomography beamline.
Setting Up and Implementing Network Data Transfer
For researchers planning to use network data transfer, the following resources are available for assistance in setting up and implementing the workflow:
- To speak with a beamline scientist who has implemented the tools described below, contact Dula Parkinson.
- To obtain and use the best equipment to build a Data Transfer Node (DTN) or for software tools such as Globus Online, contact the High Performance Computing Services Group by sending email to email@example.com
- To connect your beamline to the Lab’s fast ScienceDMZ network, or to debug networking issues at LBNL, contact firstname.lastname@example.org
- To debug national network issues, or to find contact information for offsite campus or IT groups, contact email@example.com
To Achieve Faster Data Transfer
There are three main points for users and system administrators to consider:
1) Using the right file transfer tools
Instead of FTP or scp, use tools that have been designed specifically for high-speed data transfer. We recommendGridFTP or Globus Online. GridFTP is good if you want to automate transfers, but requires significant setup. Globus Online has a graphical user interface and is easy to use. Using a fast transfer tool is the simplest thing you can do to increase data transfer speeds. LBNL extensively uses both of these transfer tools and provides an overview from the 2014 LabTech workshop, with information on how to get additional help.
2) Using capable file transfer servers
Data can only be transferred as fast as it can be read from the source disk and written to the destination disk. Most systems aren’t tuned for high speed data transfer out of the box. Systems tuned for high speed data transfer are called Data Transfer Nodes (DTNs). Beamline 8.3.2 has recently implemented such a DTN based on the reference specification provided by ESnet, which, along with a new network designed by ESnet and LBLnet, has resulted in a more than 10-fold improvement in data transfer speeds.
3) Ensuring that the end-to-end network isn’t the bottleneck
If you are using fast data transfer tools between two fast data transfer nodes, the final thing to ensure is that the end-to-end network is not impeding the transfer. This becomes even more important over long distances. The need to resend just a small amount of data can dramatically increase transfer times. Unfortunately, this can also be the most complicated area to understand and correct. There are three main areas to consider:
Use capable network switches
For big, long distance data transfers, packet loss is a significant problem. Network switches (sometimes called hubs) are a notorious cause of retransmitted data. This can happen when there are several network connections on one side of the switch that share a single connection on the other side. In this case it’s important to have switches with enough memory to store packets from one connection long enough to allow the packets from other connections to move through the switch. LBNL or home institution networking professionals can recommend good switches for your environment and scientific application.
Firewalls are a common device used to secure networks. Because they generally look at every packet that flows through them, they can create bottlenecks for big science data transfers. There is a secure, alternate approach to using firewalls commonly referred as the ScienceDMZ. It works by establishing a fast, dedicated, but secure path around the firewall. You’ll generally need one at both facilities you are transferring data between. LBNL personnel can help you use the lab’s ScienceDMZ. ESnet personnel may also be able to provide some help implementing a ScienceDMZ at your home institution. See the help contacts above.
Use a "healthy" network path
It is extremely difficult to know which network path your data is taking between LBNL and your home institution and/or whether that path is "healthy." This issue is best left to the networking professionals (see above) after ensuring that all of the critical items above are not the problem (good data transfer tools and nodes, good switches and no firewalls). While network debugging is beyond the scope of this brief article, one of the tools ESnet finds indispensable in network path analysis is perfSONAR.
Involve Your Local Experts!
If Network Data Transfer would significantly increase your productivity but you don't run your data servers yourself, please get your system and network administrators involved in the process.
Unscheduled outage resulting from power-related problem on Feb 27, 2015 beginning at 3:48 PM. As of 4:50 PM most services are now operational. More at status.lbl.gov
As reported in a January article, the IT Division is moving forward with a plan to offer an Enterprise Directory self service password reset service. The next step in the process is to allow existing lab users to register secondary contact information (a non LBL email address or a mobile phone number for text notifications). Note: all new employees and affiliates have experienced this as part of their initial account activation process since January 29, 2015.
Our soft launch of the registration process will include primarily IT employees, but we may reach out to groups we frequently work with to get feedback on the new process. This is expected to start On Thursday February 26. Feedback gathered from this effort will dictate when we launch this capability for the entire lab. In March, we will start to test the actual self-help process with a planned lab wide rollout in April.
Initial Launch Point
For those of you who also use our Windows Active Directory, the initial launch point also includes an option to reset this password, as shown below.
Our Account Management FAQ has additional details on these tools.
Register and join us for some exciting training opportunities provided by LBNL-IT!
Are you interested in learning how to write programs to get and share scientific data over the web? Looking to learn how to get started quickly with Arduino for a new project? Need to create a pivot table but don’t know how? Over the next several months the IT Division will offer a unique training schedule we hope best meets your training needs.
For more information on our course offerings see below details or visit us at:
Course Title and Description
Intro to Arduino (Hands On) | March 2 – AM Session:
Learn the basics of installing the program on your laptop; writing Arduino programs; and how to connect actuators (LEDs, motors, speakers) and control them from a program. Arduino kits will be provided.
Advanced Arduino (Hands On) | March 2 – PM Session:
You’ll be introduced to advanced Aruino concepts focusing on sensors, actuators, and programming techniques that might be used to monitor or control equipment. To attend this course we recommend you have a basic familiarity with Arduino software and hardware. Arduino kits will be provided.
Software Carpentry: Instructor Training | March 10-11:
The two-day course led by Software Carpentry founder, Greg Wilson, will introduce you to basic ideas in education psychology and instructional design. This course will provide you with an opportunity to teach your fellow scientist/engineer how to build better software to work more effectively. No previous training in teaching is required, but participants should be comfortable writing medium-sized programs and using the command line. Experience with version control tools such as Git is desirable as well.
Software Carpentry: Web Programming | March 13:
The one-day course led by Software Carpentry founder, Greg Wilson, will show you how to write programs to get, share, and syndicate data over the web, and how to write simple web applications. Participants must have previous programming experience in Python. Prior experience writing HTML is useful but not essential.
Looking for training in intermediate and advanced Excel?? We recommend the following:
- Creating Advanced Functions | March 18
- Using the “What If” Analysis Tools & Recording Macros | March 18
- Excel for Science | May 20
- Becoming a Master of Data Analysis | April 15, June 17, June 24
- Presenting Financials: Make Numbers and Statistics Standout with Excel and Powerpoint | April 15, June 17
Don’t see a class in our course offerings and would like to suggest one, or two. Feel free to drop us a note at:
We’d also love to hear how we’re doing so do share with us!
WWW, Today, and Newscenter will be under scheduled maintenance beginning at 8PM on February 2, 2015 for 4 hours and will be unavailable. All other wordpress hosted sites will also be under maintenance. No redirection will be in place. No other services will be impacted. Directory services will still be available at phonebook.lbl.gov
Additional information will available on status.lbl.gov if the maintenace window changes during the outage.
Update: Outage completed normally at ~9:10PM
Are you running out of space in your office? Are you moving to a new office? Is the size of your office being reduced? Are you responsible for dealing with the records of retiring (or already retired) scientists? Are you wondering what to do with all the files you are responsible for–which ones need to be kept, which can be archived, and which can be disposed of, all in compliance with Lab and DOE regulations? You can get the answers to these and other questions by attending a 1.5 hour workshop being put on by the Archives and Records Office.
The class is scheduled for Thusday, January 29 from 1:30 PM-3:00 PM in 50A-5132.
Go here to register: https://hris.lbl.gov/self_service/training/
Select: ITD0508 | Archiving Files & Records
The Enterprise Directory (LDAP) service is changing in January 2015. We are modifying the look of the central login page (view this link for a preview after authenticating to the web site). This new look will provide some anti-phishing advice as part of the effort and will be deployed on Thursday January 8.
We are also planning a move to a self service password reset capability modeled off successful approaches used by other institutions. This will involve pre-registration of alternate contacts (SMS text message or secondary non-lbl email address). We will be introducing this to all new affiliates and employees starting January 29 in cooperation with partners in HR. In February we will implement a soft launch within the IT Division in order to test the new registration capability before the lab wide deployment. In April, the new Password reset page (with the option of registering these alternate contacts) will be made available to anyone at the lab. This is a prerequisite for the self service approach we will be deploying in February.
At the time you are notified that your enterprise directory password will expire, you can change it as you do now. In April, if you have taken the time to register alternative contact methods, you can request a self service password reset. A one time activation password will be sent to you via an alternate method and you can use it to change your enterprise directory password.
This effort is expected to greatly improve our service - since it extends the PW reset capability beyond the Help Desk normal working hours of 8am to 5pm (and best effort staffing from 7am to 8am and 5pm to 6pm).
It's cold and rainy outside, but the reading lounge has something new to warm you up. Come check out our new energy efficient led "fire"place along with our selection of curated books, magazines, and journals.
To support testing of recently repaired emergency power off (EPO) equipment, a full network outage is scheduled on December 26 from 8:00AM-3:30PM. We expect some services to be restored before 3:30PM. All Laboratory network connectivity, most IT services, and some telephone services will be unavailable.
Impacted Services: Numerous, representative major services include:
All Laboratory Network Connectivity (except nersc.gov and es.net), but including JGI. Access to and from LBL to the internet and between subnets will be unavailable.
All Laboratory Websites excluding those hosted outside LBL. Since there will be no network connectivity to LBL, all websites hosted at LBL will be unavailable. However, www.lbl.gov, today.lbl.gov, and newscenter.lbl.gov will be available as they are hosted outside the Laboratory.
All remote access to LBL. Again, since there will no network connectivity, remote access will not be possible.
Google Services including Email (partial): During the outage, incoming and outgoing email will be queued and delivered when connectivity is restored. Users with active authenticated sessions to Google services including gmail will be able to use these services during the outage, but new authentications to Google will not be available. Access to email via IMAP will also be available, but again, outgoing and incoming email will not be delivered until after the outage.
All Authentication: No authentications using LDAP or login.lbl.gov (Shibboleth) will be possible during the outage.
All business systems will be unavailable, as there will be no network connectivity.
Phone service in buildings 46, 71 and 76
Selected Services Anticipated to be Available: