Equifax is a global information solutions company that uses trusted unique data, innovative analytics, technology and industry expertise to power organizations and individuals around the world by transforming insights into knowledge that helps make more informed business and personal decisions. Today, Equifax organizes, assimilates, and analyzes data on more than 820 million consumers and 91 million businesses worldwide.
In this article we talk to Jim Grill, Senior Director of IT Software Engineering and Automation at Equifax, and some of his team members about the company’s innovative partnership with Auburn University, aimed at bringing new automation engineers into Jim's organization.
As with every software-driven company, Equifax has challenges finding enough qualified people to fill its IT engineering positions. Jim's own situation is telling. There are significantly more developers on the application development teams than on his IT teams. It's especially difficult to find engineers who are familiar with modern approaches to infrastructure and application deployment such as automation, continuous delivery, and DevOps culture.
Working with Auburn University
As Jim and some of his colleagues reviewed the situation, they realized that, no matter what, hiring is difficult and that no matter who you hire, you have to train them. "We decided to focus more on the type of people we wanted,” Jim says “We want people who are passionate about learning new things and who love what they do. Given that, and that we were willing to train whoever we hired, it made sense to start looking at universities. We had the idea to set up an office near a university that would make it easy to attract graduates and interns. As it happens, we had a connection with Auburn University. My boss is an alumnus, still lives in the area, and knows quite a few people there. Even better, Auburn is only a two-hour drive from Equifax campuses in the Atlanta area.
"The school helped a lot. They loved the idea. They're always looking for a major employer in Auburn. It's a huge deal. They also had a space we could lease that's in the university airport.
"Of course, nothing happened over night. It took about a year to put all this together. During that year, before we got up to speed, we had several people from our Atlanta office go to the university's career days to engage with people. During that year we secured the office space and developed it into an open office format that is conducive to collaboration and hired our first wave of employees. We officially opened our new Auburn IT talent center in December, 2015 and through our connections we had with the school, we sponsored a hack day to attract attention and talent.
"Since then, everything seems to be happening really fast. From January to now we've built up a group of just over 30 people. Two of them had prior experience and are team leaders. Almost everyone else is a recent Auburn graduate. We even have a few who are still going to school part time."
Of those people, half of them are primarily focused on Chef. They spend 50 percent or more of their time working on Chef, the software deployment pipelines, and release engineering. The others are still involved with Chef but are focused on other responsibilities. (To learn more about how Equifax uses Chef as part of its release pipeline, see Using Chef at Equifax.)
The initial training
Jim and his team have developed a training program for their new recruits that closely follows the training available from Chef. "When you plan on hiring 30 or more people in a short amount of time, you need a plan," Jim says. "It was a challenge when the first employees started. We quickly realized that, while it was a good idea to bring in new talent, we really had not figured out what that meant in terms of them being ready for assignments and the impact on our teams yet, so they sat around for a while without anything to do. That motivated us to fix the process. We didn't want that to happen again and again with every new hire that came in.
"Now, if they start before we're ready to begin formal training, we have lots of documentation to read and they also need to read The Phoenix Project. We'll also have them set up their laptops and make sure they can create OpenStack VMs.
"By the time we're ready for them, they will already have learned a lot. Then, they go through our Chef training classes and an abbreviated version of our boot camp. The only difference is they don't end up building their own pipeline. They use whichever one belongs to the team they're on. We're now at the point where teams are doing this training themselves. It's not necessarily one of my teams. Of course, we're always there if there are questions or problems. (To learn more about how Equifax trains its development and operations teams, see Chef and DevOps at Equifax.)
Emily Traylor, Automation Team Lead, says, “It’s refreshing to see how far the training program has come in its first year. In the beginning, the training was more ad hoc, with a lot of questions from each new team member. Sometimes I didn’t know the ‘right’ questions to ask, but the team was always helpful. Now, the training program is more structured; the new recruits have a training calendar for their first few weeks, with specific courses to take, both technical and non-technical. Since so many of our new recruits are recent college grads, and this is their first job, we train them in Chef, Python, Git, Bitbucket, and other technologies, but we also train them in how to work and how to be successful in their first job.”
Jonathan Reeves, Automation Engineer, added, "Compared to my previous job experience, Equifax does a phenomenal job of training, providing resources, and collaborating in team environments to make the most of Chef and other automation tools that we regularly use. From day one, we were provided with instructions and resources to teach us to set up a development environment and begin experimenting with Chef. Once I was up to speed, I joined one of the development teams and we mastered a rhythm of completing tasks soon after.”
Learning by doing
The Auburn recruits become members of existing teams. These teams realize that, because their new team members need help and training, everyone must be committed to working with them and teaching. They also realize that, in the long run, the effort is well worth it.
"As should be true of any team, at no point is anyone solely responsible for anything," says Jim. “I don't think it ever crossed our minds that these people aren't ready, they don't have experience. You put people on a team and that team should work in an open, transparent way. All of us are using one Agile methodology or another. Teams are either Kanban or Scrum, depending on the kind of work that they do. One of my teams is primarily focused on long-term projects where everything is planned, with potentially years of work and no end in sight. Another team does general automation and because things are happening all the time, services that they operate break, recipes fail, whatever it is, they're responsible for fixing it and helping people.
"We have very open work practices. Everybody knows what's happening, it's on the board, everybody knows who's doing what. I don't have to have status update meetings with my leaders to find out where things are. Anybody can look at these boards. I think, with that kind of environment, it's safe to bring people onto a team when they don't have a lot of experience or knowledge because they're never alone.
"We consider ourselves to be teaching teams and we provide environments for learning. We can't always do this but, generally, what we try to do is, when we're pulling things off the Kanban board or the Scrum board, the person with the least skills in that area gets that task.
"If you know absolutely nothing about something and there's a card up on the board for it, that's the card you take! You don't take a card that's squarely in your wheelhouse. You take something you're not qualified to do and you get help from somebody else. Everybody knows that it's their job to help someone else accomplish a task they're not qualified to do alone. Doing that, we're a little bit slower but, at the end of the day, people become SMEs in areas you'd least expect. We’ve been rewarded by developing some very surprising experts. We've got a guy who comes from a software engineering background who's now our expert on patching. He knows everything there is to know about our automated patching system. I never would have guessed that six months ago he would be that person!
"With us, no one's ever alone; they've always got the team behind them. Any team that fails, fails together. Any team that wins, wins together."
Emily comments, "One of the things I like most about working at Equifax is that we have a blameless culture. We celebrate failures, as long as we ‘fail faster’, so that we can improve faster. No matter what, I know I have the support of my team and that management always has my back. Even though everyone may not always have the best idea, we encourage creativity because that is when the great ideas really come out."
Drew Tewell, Release Engineer, enjoys the chance to learn. "Equifax really supports a learning environment. Through tutorial classes and documentation, new employees, like myself, are given the context and knowledge they need to find creative and innovative solutions to the problems we face. By encouraging employees to take on new challenges and not be afraid to fail, they allow developers to learn new skills and grow as professionals, benefiting both the company and the individual.”
New employee Phillip Ferentinos, Automation Engineer, says, "As a recent college grad, I was really nervous I'd be working with a technology I had never seen before. That's exactly what happened. Thankfully, our team is set up to so that if you don't know how to do something, you're the first one to do it while someone guides you. That way, the next time someone joins the team, you can guide them.”
Bringing the teams together
Equifax uses a combination of technology and face-to-face interactions to keep the Auburn and Atlanta offices working together. "Constant communication is the only way this works," Jim says.
“It would be much easier if we were all co-located together. Remote takes effort on both sides. We travel a lot. At least every other week we've got somebody here or there, traveling back and forth. We cycle people in various ways. Sometimes we bring the whole Auburn group out to Atlanta. Sometimes we bring five or six people out. We also like to bring people out individually because we want the Auburn people to know us.
"If we bring six of them out at once, they all sit together, they all go to lunch together, they all stay at the same hotel, hang out with each other. We want the Auburn people to hang out with the Atlanta teams, go to lunch with them, work with them. Of course, it goes both ways. We have Atlanta people who drive to Auburn and work there for three or four days.
"We use HipChat to stay in contact all day and we also use a couple A/V systems. These systems have a high-definition camera and a high-definition TV on a cart. The remote cameras can be controlled from either site, so you can do things like zoom in or pan left or right. These systems also have a noise-canceling speaker you can put on a table or desk. These systems are great for sprint planning and other team meetings.
"A lot of times we'll have a less experienced person in Auburn doing the work while someone in Atlanta watches. We do screen shares, and can take control of the desktop, but the idea is to share a screen, sit there, and talk over headphones. The least qualified person does the work while the most qualified person watches and helps.
I think everybody enjoys it. We meet regularly to find out what we're doing right or wrong. We do this in our retrospectives, of course, it's part of our Agile process, but we also do this just in general. It's my job to talk to the managers that we have on the other end and ask them, 'How are we doing? What are people saying?' I have one-on-ones with them as well to find out what we could do better. We take that feedback and make sure that we remind everybody that they need to put forth the effort. People don't mean to ignore each other but they get busy and they forget."
Jim reflected on what he and his group have accomplished so far. "In terms of the success of the program, I'd say that, at first, it's slow. Any time you're training people, it's difficult. A lot of times, even with a well-established team, even if you bring in somebody with over 20 years of experience, it's still disruptive. That person is drinking from the fire hose for the first 60 to 90 days. It's not until a year that you hit your stride and learn everybody's names in other departments and know who to ask if you have questions or need help. That's when you can be truly productive.
"Imagine bringing on three or four people with little to no experience to a team that has no more than seven to ten people. It's hard. It's the longer term that must be your focus. You're not as concerned with the short term. You're marching towards a target that's way out there. It's important that everyone keeps sight of that vision and understands what the goals are. That's how you get your teams to make such a big commitment. It’s absolutely critical to our success that we scale our practices in terms of automation, and that’s exactly what we’re doing. This program is an absolute success!"
Jim then talked about what he'd like to see in the future. "I'm not sure about the details. I do know that our program is successful and we're planning to expand. I'd like to see us do something similar with other nearby universities.
"Hiring, they say, is one of the most important things you do as a manager. I think it is. You can't get anything accomplished without people and you can't get anything great accomplished without great people. You're not going to find somebody who can do everything you want them to do. It just doesn't happen. It's our responsibility to teach and train these people and provide the environment to help them be great. It makes a lot of sense to me to also be very good at that. That should also be the most important thing that we're doing because, like I said, you need great people to do great things, and none of our plans are mediocre. I consider everything on my plate to be absolutely critical to our success. There's nothing where a so-so job is good enough. Everything's important. We want to be great."