Exploring AWS CodeGuru: New automated code review has smart features – but Java-only

Do the sums before adopting

Analysis AWS's automated code review and profiling tool CodeGuru drew substantial interest when it was announced at the vendor's re:Invent shindig in Las Vegas this month.

Developers like the idea of automated code review and even small amounts of time saved or bugs averted are of considerable value. Modern editors like Eclipse or Visual Studio Code already provide coding assistance with common errors, auto completion and auto formatting; if you try coding in Notepad without this kind of help you soon discover how useful it is.

CodeGuru, needless to say, promises the magic of machine learning, based on "hundreds of thousands of internal projects" as well as "over 10,000 open source projects on GitHub."

There is also CodeGuru Profiler, bundled with it but really a separate tool, which helps to identify expensive lines of code. Profiling is nothing new, but it can help identify inefficiencies that yield big improvements when fixed. In a presentation at re:Invent, attendees were told of examples where the number of instances required to run an application reduced by 75 per cent after optimising a thread contention issue, or by between 40 per cent and 67 per cent in another case.

CodeGuru is in preview and we took a quick look. There are a number of limitations. Both the code review and the profiler only work on Java applications. The code review only supports either GitHub or AWS CodeCommit repositories, and the profiler requires an application hosted on Amazon EC2, or containerised on ECS (Elastic Container Service) or Fargate.

The CodeGuru process: how it is meant to work

The CodeGuru process: ML applied to code review

El Reg gives it a spin

Getting started with Reviewer is a matter of signing into AWS, heading to the CodeGuru dashboard, and "associating" a GitHub repository. This worked fine for us when signing into GitHub as the repository owner, but the documentation recommends creating a dedicated CodeGuru user on GitHub, since the reviews are posted as comments by the account used for association, which can be confusing.

We tried doing this by setting up a separate GitHub user as collaborator on the project, but then the repository association failed. It probably works OK with a GitHub organisation, which is the more common scenario in business. What was annoying, though, was that we could not remove the failed association ("Disassociate" did not work), nor add it again, so we had to resort to creating a new repository to test CodeGuru.

CodeGuru says it is analyzing our code but not finding any recommendations

CodeGuru says it is analyzing our code but not finding any recommendations

CodeGuru is triggered by pull requests, which is when a developer asks for their code to be merged into the repository. We used a simple Spring Boot Java project and started making pull requests. Unfortunately we could not persuade CodeGuru to come up with any recommendations. According to the documentation it provides recommendation on AWS best practice, concurrency, resource leaks, sensitive information leaks, and common coding best practice. We tried some obviously buggy code:

MyClass c = new MyClass();
c = null;
return c.GetMessage();

And when that passed unnoticed, a simple race condition, but while CodeGuru said it had analysed the code, there were no recommendations.

How CodeReview is meant to work

How CodeReview is meant to work

Well, it is a preview and we are not alone; another user reported the same issue in the AWS forums. Support can no doubt work it out. You can see what is meant to happen in the presentation here. Presuming it works, though, how much does it cost? The price is $0.75 per 100 lines of code, which could work out to be expensive. In Amazon's pricing example, 50 developers making 600 pull requests of 200 lines of code each comes to $900 a month. However, if a substantial pull request was reworked a few times, the amount of code could easily exceed this estimate.

The CodeGuru Profiler

The CodeGuru Profiler

The pricing for the profiler is $0.005 per sampling hour up to a maximum of 36,000 hours per month, so the maximum charge is $180 per application. While this may soon pay for itself after finding a needed optimisation, it may be something you want to turn off once an application is running sweetly. You would also want to be convinced that the CodeGuru profiler was finding things that other more cost-effective profilers miss. The AWS claim is that the profiler easily identifies the lines of code that need to be optimised, and that it can run in production "without impact to the application."

The profiler has three components. There is an agent which you add to your application, a profiler service, and a recommendation engine. The results are presented as interactive visualisations. There are three types of visualisation: an overview, a "Hotspots" visualisation showing the functions which consume the most time, and an "Inspect" visualisation that shows code grouped with the functions it calls, and the functions by which it is called.

The good news is that AWS CodeGuru is offered with a free 90-day trial. That is long enough both to profile some existing applications, and to try the CodeGuru Reviewer in anger to see if it is delivering enough value to justify its cost. Otherwise, these are not cheap services, and we suggest careful evaluation. ®

Similar topics

Narrower topics

Other stories you might like

  • Cheers ransomware hits VMware ESXi systems
    Now we can say extortionware has jumped the shark

    Another ransomware strain is targeting VMware ESXi servers, which have been the focus of extortionists and other miscreants in recent months.

    ESXi, a bare-metal hypervisor used by a broad range of organizations throughout the world, has become the target of such ransomware families as LockBit, Hive, and RansomEXX. The ubiquitous use of the technology, and the size of some companies that use it has made it an efficient way for crooks to infect large numbers of virtualized systems and connected devices and equipment, according to researchers with Trend Micro.

    "ESXi is widely used in enterprise settings for server virtualization," Trend Micro noted in a write-up this week. "It is therefore a popular target for ransomware attacks … Compromising ESXi servers has been a scheme used by some notorious cybercriminal groups because it is a means to swiftly spread the ransomware to many devices."

    Continue reading
  • Twitter founder Dorsey beats hasty retweet from the board
    As shareholders sue the social network amid Elon Musk's takeover scramble

    Twitter has officially entered the post-Dorsey age: its founder and two-time CEO's board term expired Wednesday, marking the first time the social media company hasn't had him around in some capacity.

    Jack Dorsey announced his resignation as Twitter chief exec in November 2021, and passed the baton to Parag Agrawal while remaining on the board. Now that board term has ended, and Dorsey has stepped down as expected. Agrawal has taken Dorsey's board seat; Salesforce co-CEO Bret Taylor has assumed the role of Twitter's board chair. 

    In his resignation announcement, Dorsey – who co-founded and is CEO of Block (formerly Square) – said having founders leading the companies they created can be severely limiting for an organization and can serve as a single point of failure. "I believe it's critical a company can stand on its own, free of its founder's influence or direction," Dorsey said. He didn't respond to a request for further comment today. 

    Continue reading
  • Snowflake stock drops as some top customers cut usage
    You might say its valuation is melting away

    IPO darling Snowflake's share price took a beating in an already bearish market for tech stocks after filing weaker than expected financial guidance amid a slowdown in orders from some of its largest customers.

    For its first quarter of fiscal 2023, ended April 30, Snowflake's revenue grew 85 percent year-on-year to $422.4 million. The company made an operating loss of $188.8 million, albeit down from $205.6 million a year ago.

    Although surpassing revenue expectations, the cloud-based data warehousing business saw its valuation tumble 16 percent in extended trading on Wednesday. Its stock price dived from $133 apiece to $117 in after-hours trading, and today is cruising back at $127. That stumble arrived amid a general tech stock sell-off some observers said was overdue.

    Continue reading

Biting the hand that feeds IT © 1998–2022