L&L Conference Recap: Key Takeaways on AI, Validation, and the Future of Digital Forensics
Author: Frank Hammock
Digital Forensics Specialist
Oxygen Forensics just recently put on their Legacy and Logic conference in Orlando FL, celebrating 25 years of consistent innovation in the industry. This was an educational event, with many of the industry's most experienced members sharing their expertise. ArcPoint was lucky enough to be invited to present, and I had the honor of getting on stage with Amy Quezada. Being among the presenters was definitely a thrill for my first conference, but members of Oxygen Forensics such as Keith Lockhart made the entire process a breeze.
Our presentation dove into some of the biggest roadblocks examiners run into in their investigations, and how we believe that triage and automation can throw some punches against the industry's constant backlog problem. The modern data explosion was a common theme in most presentations, and every attendee had their own thoughts on what changes are needed. It was clear that the common division in thought was on the use of AI to tackle this problem. The L&L conference consisted of two days of presenting, and Amy and I were lucky enough to present on the first day, so I was able to sit in on many of the other presentations and learn from the best. There was at least one slide on AI in 90% of the presentations I attended, and 2-3 that had the topic as the main event. It’s not shocking that AI use was the most common talking point, with it exploding across the world, but it was very interesting to see the equally violent opposing sides.
At only 20 years old, I was the youngest in the conference by a good margin, but I would like to think of it as a built-in icebreaker when it came to gaining knowledge from the veterans. I would ask 2 main questions among others, but the big 2 were as follows: “How did you get into the digital forensics field?” and “Where does your operation struggle the most?” These questions jump-started plenty of interesting conversations from different perspectives, especially with the different scopes of the businesses. There were plenty of smaller operations, and even one-person shows, along with some employees from the big organizations. Many of the examiners I talked to were former law enforcement officers, litigation specialists, and software developers. But there answers to where their operations struggled the most were very interesting to me. Plenty talked about their headaches keeping up with iOS and MacOS, and consistent encryption use. But many were struggling logistically, with rising transportation and material costs. When an outsider thinks about the digital forensics industry, the everyday issues that may come to mind are software-based, or “how would they get into my photos if my computer is locked?” But even we professionals can lose sight of the traditional problems that every business faces. The products and services may be great, but simple supply chain issues are still hampering many people in our industry. I would like to thank the “mandated” cocktail hour built into the conference for many of these conversations, as this time was the loosest for obvious reasons. As I am underage, I was happy to talk to my older counterparts with a catered meatball in one hand and a Sprite in the other. I am thankful that the attendees did not take my beverage of choice as a sign of inexperience, and still took ample time to discuss some of the hidden parts of the industry.
L: Amy Quezada
R: Frank Hammock
One of my favorite events of the conference was the discussion of AI use between examiners Alexis Brignoni & Scott Tucker. Both are very accomplished in their respective careers, but do not quite see eye to eye on AI. Brignoni made his stance very clear at the beginning of their hour segment by quoting Studio Ghibli legend Hayao Miyazaki, “AI is an insult to life itself.” Tucker, on the other hand, spoke about his daily AI use, and reaffirmed its purpose as a tool and not something to be trusted on its own. The two veterans of the industry ended up agreeing more on the topic than I initially thought they would, with Brignoni leaning more towards very limited use, and Tucker as a moderate user. But two things were at a very clear consensus: AI cannot be trusted on its own, and that examiners expertise needs to validate results. AI is not marketed this way to the public, but giving an LLM the keys was against almost every attending professional's wishes. So in my opinion, AI needs to be taught this way, as a tool and not a miracle machine. Seasoned examiners are able to use it properly because they have years of experience and prior knowledge to validate the AI model, but it could be risky to allow Junior analysts unrestricted access when they simply may not have the experience to validate properly.
Sam Brothers dove deeper into investigations of the use of AI with a large focus on admissibility in court, and reaffirming the importance of our industry. Brothers definitely was concerned about the potential use, mostly because of AI’s accuracy rate. He talked about how in other industries it may be okay for a tool to only be right 90% of the time, but in forensics there is too much at stake to rely on those numbers. We are deciding whether or not someone's life will be uprooted, and whether or not potential victims will get the justice they deserve. I agree with his concern, we do not have the luxury of making mistakes, and that is why AI cannot replace the expertise of examiners. But I do think that AI can be utilised in a way that aids examiners. Brothers talked about some of the pros later in his presentation which I agreed with. AI is great at handling massive data sets, and is very good at pattern recognition. AI is built for busy work such as hashing, parsing, and reporting. It can give standardized versions of data that eliminates redundant human steps to normalize data. All of these steps need validation, but so does almost everything else in DFIR. Check your work always, but AI is built to streamline your process. It is not built to be trusted on its own, and that is where most problems come from, users giving AI the authority to act on its own.
The final presentation of the conference was given by John Bradley, who served as an expert witness for the prosecution in the famous Casey Anthony trial. Bradley is the founder of API Forensics and has been in the forensics and software development world for over 30 years. For the uninitiated, the State of Florida v. Casey Marie Anthony was a murder trial where Anthony was charged with the murder of her 3 year old daughter, Caylee. Caylee had traces of chloroform and duct tape near the body and in the car allegedly used to transport her, so the prosecution's working theory was that Casey killed her daughter by duct taping her airways and administering chloroform. John Bradley worked with law enforcement to recover deleted internet history from unallocated space using his own program named CacheBack. Based on the program, it revealed that “chloroform” had been searched 84 times on the Anthony home computer. But it was revealed later that this number was wrong due to a bug in Bradley’s program. When reliving some of the trial in front of us, Bradley passionately hammered in the importance of multiple tool validation. Florida law enforcement had actually used a separate tool to get the deleted history that directly contradicted CacheBacks report. The prosecution then submitted both reports, with one tool stating chloroform was only searched one time, and CacheBack stating it was searched 84 times. Validation is one of if not the most important processes when dealing with digital evidence. If there was a proper attempt to validate the deleted search history, the CacheBack bug could have been fixed, and one of the largest holes in the prosecution could have been covered. Along with validation, Bradley also went over some general tips on taking the stand, and how to prepare properly. Noting multiple flaws in the prosecution's preparation, including the distinct lack of notes from other witnesses, and an expert witness that did not give his own opinion, but simply stated facts.
After the two days of presentations, Oxygen offered a day of training led by Amanda Mahan that I was lucky enough to attend. She walked me and around 20 other attendees through Oxygen Detective and its features, and was very knowledgeable and responsive to any of our questions or concerns about the product.
Overall, for my first conference I have nothing but good reviews. I am worried I may have been spoiled early on, as it is hard for me to imagine a better environment for like minded individuals to share their experiences and knowledge with attendees willing to learn. I would really like to thank Keith Lockhart , Lee Reiber, and the rest of Oxygen Forensics for putting on such a wonderful event. I loved the opportunity to speak so early in my career, and I loved all of the productive conversations that came from it.
Follow @arcpointforensics to stay updated on the most recent DFIR news and ATRIO updates!
