Programming Problems & Solutions : “Exploring IP Address Ranges in Go”. The introduction to this series is here and includes all links to every post in the series. This is the 8th of a dozen programming challenges I’m doing to setup for an eventual blog post on AI coding. The intent is to setup these programming challenges, get a solution, find a good refactoring, and then see how the AI tooling performs going through the same thing or refactoring what is in place. If you’re interested in how AI performs, and checking out these experiments and tests, subscribe to the blog to have the articles delivered directly to your email!

Hey there, fellow code adventurers! Today, we’re diving into the world of IP addresses and ranges. We’ve got an exciting coding challenge on our hands, and we’ll be tackling it using the Go programming language. So, grab your favorite beverage, and let’s get started!

The Challenge

Our mission, should we choose to accept it, is to implement a function that takes two IPv4 addresses as input and returns the number of addresses between them (including the first one, but excluding the last one). We’ll be working with valid IPv4 addresses in the form of strings, and the last address will always be greater than the first one.

Examples To give you a better idea of what we’re aiming for, here are a few examples:

• With input “10.0.0.0” and “10.0.0.50”, the function should return 50.
• With input “10.0.0.0” and “10.0.1.0”, the function should return 256.
• With input “20.0.0.10” and “20.0.1.0”, the function should return 246.

Testing Our Code Before we dive into the implementation, let’s take a look at the tests we’ll be running to ensure our code is up to par. We’ll be using the Ginkgo testing framework and the Gomega assertion library to write our tests.

package main

import "testing"

func TestIpsBetween(t *testing.T) {
	t.Run("should handle basic cases", func(t *testing.T) {
		assertEqual(t, IpsBetween("10.0.0.0", "10.0.0.50"), 50)
		assertEqual(t, IpsBetween("20.0.0.10", "20.0.1.0"), 246)
	})

	t.Run("should handle large ranges", func(t *testing.T) {
		assertEqual(t, IpsBetween("10.0.0.0", "10.0.1.0"), 256)
		assertEqual(t, IpsBetween("10.0.0.0", "10.1.0.0"), 65536)
	})

	t.Run("should handle different octets", func(t *testing.T) {
		assertEqual(t, IpsBetween("10.0.0.0", "11.0.0.0"), 16777216)
		assertEqual(t, IpsBetween("10.0.0.0", "12.0.0.0"), 33554432)
	})

	t.Run("should handle same start and end addresses", func(t *testing.T) {
		assertEqual(t, IpsBetween("10.0.0.0", "10.0.0.0"), 0)
		assertEqual(t, IpsBetween("192.168.1.1", "192.168.1.1"), 0)
	})
}

func assertEqual(t *testing.T, actual, expected int) {
	if actual != expected {
		t.Errorf("Expected %d, but got %d", expected, actual)
	}
}

These tests cover various scenarios, including basic cases, large ranges, different octets, and even the case where the start and end addresses are the same.

The Solution Now, let’s take a look at the solution:

package main

import (
	"fmt"
	"net"
)

func main() {
	fmt.Println(IpsBetween("10.0.0.0", "10.0.0.50"))
}

func IpsBetween(start, end string) int {
	startIP := net.ParseIP(start).To4()
	endIP := net.ParseIP(end).To4()

	if startIP == nil || endIP == nil {
		panic(fmt.Sprintf("Invalid IPv4 address: start=%s, end=%s", start, end))
	}

	startNum := ipToInt(startIP)
	endNum := ipToInt(endIP)

	return int(endNum - startNum)
}

func ipToInt(ip net.IP) uint32 {
	result := uint32(ip[0]) << 24
	result |= uint32(ip[1]) << 16
	result |= uint32(ip[2]) << 8
	result |= uint32(ip[3])
	return result
}

Let’s break it down:

1. We start by parsing the input IPv4 addresses using net.ParseIP() and converting them to 4-byte representations using .To4(). This ensures we’re working with valid IPv4 addresses.
2. If either startIP or endIP is nil, it means the input address is invalid, so we panic with an appropriate error message.
3. We convert the start and end IP addresses to their integer representations using the ipToInt() function. This function shifts and combines the individual bytes of the IP address to form a single 32-bit unsigned integer.
4. Finally, we calculate the difference between the integer representations of the end and start IP addresses and return the result as an int.

And there you have it! A solution to calculate the number of IP addresses between two given IPv4 addresses.

Refactoring

Ok, I kicked off and came up with some good refactoring, here are the changes:

1. In the IpsBetween function, I directly return the result of subtracting the int representations of the start and end IP addresses. I parse and convert the IP addresses inline using net.ParseIP(end).To4() and net.ParseIP(start).To4().
2. Next I removed the error handling with the panic statement. If an invalid IP address is provided, net.ParseIP() will return nil, and the ipToInt function will panic with a “runtime error: invalid memory address or nil pointer dereference” error. This is a more idiomatic way of handling errors in Go, as it allows the caller to handle the error appropriately.
3. In the ipToInt function, I combined the bit shifting and OR operations into a single line to make it more concise.

With these changes, the code is more compact and easier to read. The functionality remains the same, and it still passes all the tests. It comes out looking like this

package main

import (
	"fmt"
	"net"
)

func main() {
	fmt.Println(IpsBetween("10.0.0.0", "10.0.0.50"))
}

func IpsBetween(start, end string) int {
	return int(ipToInt(net.ParseIP(end).To4()) - ipToInt(net.ParseIP(start).To4()))
}

func ipToInt(ip net.IP) uint32 {
	return uint32(ip[0])<<24 | uint32(ip[1])<<16 | uint32(ip[2])<<8 | uint32(ip[3])
}

I hope you enjoyed this coding adventure as much as I did. Until next time, happy coding, and may your IP addresses always be in range!

References

• Github repo and the first draft followed with the refactored code.

Programming Problems & Solutions : “Transforming 2D Arrays to CSV Format in Go”. The introduction to this series is here and includes all links to every post in the series. This is the 7th of a dozen programming challenges I’m doing to setup for an eventual blog post on AI coding. The intent is to setup these programming challenges, get a solution, find a good refactoring, and then see how the AI tooling performs going through the same thing or refactoring what is in place. If you’re interested in how AI performs, and checking out these experiments and tests, subscribe to the blog to have the articles delivered directly to your email!

Hey there, coding enthusiasts! Today, I’m diving into an interesting problem that involves converting a two-dimensional numeric array into its CSV (Comma-Separated Values) representation. This is a common task when working with data in various formats, and I’ll explore how to tackle it using Go.

The Problem: Imagine you have a 2D array filled with numbers, and you need to convert it into a CSV string. Each row of the array should be represented as a line in the CSV, with the elements separated by commas. For example, consider the following input array:

[
    [0, 1, 2, 3, 4],
    [10, 11, 12, 13, 14],
    [20, 21, 22, 23, 24],
    [30, 31, 32, 33, 34]
]

The expected output CSV string should look like this:

0,1,2,3,4
10,11,12,13,14
20,21,22,23,24
30,31,32,33,34

Before I jump into the implementation, I’ll define some tests to ensure our solution works as expected. I’ll use the standard Go testing package and create a test file named csv_test.go. Here’s what my tests look like:

package main

import "testing"

func TestToCsvText(t *testing.T) {
	testCases := []struct {
		input    [][]int
		expected string
	}{
		{
			input: [][]int{
				{0, 1, 2, 3, 45},
				{10, 11, 12, 13, 14},
				{20, 21, 22, 23, 24},
				{30, 31, 32, 33, 34},
			},
			expected: "0,1,2,3,45\n10,11,12,13,14\n20,21,22,23,24\n30,31,32,33,34",
		},
		{
			input: [][]int{
				{-25, 21, 2, -33, 48},
				{30, 31, -32, 33, -34},
			},
			expected: "-25,21,2,-33,48\n30,31,-32,33,-34",
		},
		{
			input: [][]int{
				{5, 55, 5, 5, 55},
				{6, 6, 66, 23, 24},
				{666, 31, 66, 33, 7},
			},
			expected: "5,55,5,5,55\n6,6,66,23,24\n666,31,66,33,7",
		},
	}

	for _, tc := range testCases {
		result := ToCsvText(tc.input)
		if result != tc.expected {
			t.Errorf("Expected: %q, Got: %q", tc.expected, result)
		}
	}
}

I then define a test function TestToCsvText that includes multiple test cases. Each test case consists of an input 2D array and the expected CSV string. I iterate over the test cases, call my ToCsvText function with the input, and compare the result with the expected output. If there’s a mismatch, the test fails, and an error is reported.

Now, let’s take a look at the implementation of the ToCsvText function in the csv.go file:

package kata

import "strconv"

func ToCsvText(array [][]int) string {
    var result string

    for i, row := range array {
        for j, num := range row {
            result += strconv.Itoa(num)
            if j < len(row)-1 {
                result += ","
            }
        }
        if i < len(array)-1 {
            result += "\n"
        }
    }

    return result
}

A run of the tests and victory, first draft is good to go!

The ToCsvText function takes a 2D slice of integers ([][]int) as input and returns the CSV string representation. Here’s how it works:

1. I initialize an empty string variable result to store the CSV output.
2. I use a nested loop to iterate over each row and each element of the 2D array.
3. For each element, I convert the integer to a string using strconv.Itoa(num) and append it to the result string.
4. After appending each element, I check if it’s not the last element in the row. If it’s not the last element, I append a comma (,) to separate the elements.
5. After processing each row, I check if it’s not the last row. If it’s not the last row, I append a newline character (\n) to separate the rows.
6. Finally, I return the result string, which contains the CSV representation of the 2D array.

With that, I’ve got it! However as always, I like to take a stab at some refactoring. Get the code whittled down to something more concise or get it into a more performant state.

Refactoring

Let me start by revisiting the original implementation of the ToCsvText function. While this implementation works, I can simplify it by leveraging Go’s built-in strings.Join function and avoiding the manual concatenation of commas and newline characters. Here’s my refactored version:

func ToCsvText(array [][]int) string {
    var rows []string

    for _, row := range array {
        var rowStr []string
        for _, num := range row {
            rowStr = append(rowStr, strconv.Itoa(num))
        }
        rows = append(rows, strings.Join(rowStr, ","))
    }

    return strings.Join(rows, "\n")
}

Now for a few more tests.

        // ... (existing test cases)
        {
            input:    [][]int{},
            expected: "",
        },
        {
            input: [][]int{
                {1},
            },
            expected: "1",
        },
        {
            input: [][]int{
                {1, 2, 3},
                {4, 5, 6},
            },
            expected: "1,2,3\n4,5,6",
        },

I’ve added the following:

1. An empty input array ([][]int{}) to ensure my function handles an empty input gracefully and returns an empty string.
2. A single-element array ([[1]]) to verify that my function correctly handles a 2D array with only one element.
3. A 2D array with multiple rows and columns ([[1, 2, 3], [4, 5, 6]]) to confirm that my function generates the expected CSV string for a more complex input.

Again, another good build and tests passing.

Until next coding challenge, happy thrashing code!

Reference

• Github repo and first draft followed with the refactored code.

Programming Problems & Solutions : “How to Convert an IPv4 Address to a 32-bit Integer in C#: A Step-by-Step Guide”. The introduction to this series is here and includes all links to every post in the series. If you’d like to watch the video (see just below this), or the AI code up (it’s at the bottom of the post) they’re available! But if you just want to work through the problem keep reading, I cover most of what is in the video plus a slightly different path down below.

Hey there, fellow code wranglers! Time to dive deep into the land of bits and bytes and IP addresses. Today, I’m tackling a neat little programming challenge: converting an IPv4 address into a 32-bit integer. A fun exercise that hones your bitwise operation skills and deepens your understanding of IP addresses.

Let’s kick things off with the essentials. An IPv4 address, such as 128.32.10.1, consists of four octets. Each octet is a byte (8 bits), and together they form a 32-bit number. Here’s a quick breakdown:

• 1st octet: 128 -> 10000000
• 2nd octet: 32 -> 00100000
• 3rd octet: 10 -> 00001010
• 4th octet: 1 -> 00000001

When we line them up, we get: 10000000.00100000.00001010.00000001

This binary sequence can be translated to the 32-bit number 2149583361. Our task is to write a function ToInt32 in C# that takes an IPv4 address as input and returns this 32-bit number.

Let’s break it down step-by-step:

1. Split the IP Address: We split the string representation of the IP address into its four octets.
2. Convert to Integer: Convert each octet from a string to an integer.
3. Bitwise Shifting: Shift each octet into its proper position within the 32-bit integer.
4. Combine Using Bitwise OR: Use the bitwise OR operation to combine these values into the final 32-bit integer.

Here’s how we do it in code:

namespace IpConverter;

public static class IpConvert
{
    public static uint ToInt32(string ip)
    {
        string[] octets = ip.Split('.');
        uint ipNumber = 0;

        ipNumber += Convert.ToUInt32(octets[0]) << 24; // Shift the first octet left by 24 bits
        ipNumber += Convert.ToUInt32(octets[1]) << 16; // Shift the second octet left by 16 bits
        ipNumber += Convert.ToUInt32(octets[2]) << 8;  // Shift the third octet left by 8 bits
        ipNumber += Convert.ToUInt32(octets[3]);       // Fourth octet stays as is

        return ipNumber;
    }
}

Walking Through the Code:

using IpConverter;

namespace IpConverterTests;

public class Tests
{
    [SetUp]
    public void Setup()
    {
    }

    [Test]
    public void TestIPtoInt()
    {
        Assert.That(IpConvert.ToInt32("128.32.10.1"), Is.EqualTo(2149583361), "Incorrect answer for ip = \"128.32.10.1\"");
    }
}

This test checks whether the function ToInt32 correctly converts "128.32.10.1" to 2149583361. When the test passes, we can confidently say our function works for this input. At this point, this seems to be a working solution, but I’m going to add a few more tests regardless, just to have them cover more of the cases to make sure my refactoring doesn’t get off in the weeds.

Refactoring

First up, the tests I added.

public class Tests
{
    [Test]
    public void TestIPtoInt_ValidIP()
    {
        Assert.That(IpConvert.ToInt32("128.32.10.1"), Is.EqualTo(2149583361), "Incorrect answer for ip = \"128.32.10.1\"");
    }

    [Test]
    public void TestIPtoInt_AllZeros()
    {
        Assert.That(IpConvert.ToInt32("0.0.0.0"), Is.EqualTo(0), "Incorrect answer for ip = \"0.0.0.0\"");
    }

    [Test]
    public void TestIPtoInt_AllOnes()
    {
        Assert.That(IpConvert.ToInt32("255.255.255.255"), Is.EqualTo(4294967295), "Incorrect answer for ip = \"255.255.255.255\"");
    }

    [Test]
    public void TestIPtoInt_LeadingZeros()
    {
        Assert.That(IpConvert.ToInt32("001.002.003.004"), Is.EqualTo(16909060), "Incorrect answer for ip = \"001.002.003.004\"");
    }

    [Test]
    public void TestIPtoInt_RandomIP()
    {
        Assert.That(IpConvert.ToInt32("192.168.0.1"), Is.EqualTo(3232235521), "Incorrect answer for ip = \"192.168.0.1\"");
    }

    [Test]
    public void TestIPtoInt_RandomIP2()
    {
        Assert.That(IpConvert.ToInt32("10.0.0.1"), Is.EqualTo(167772161), "Incorrect answer for ip = \"10.0.0.1\"");
    }
}

• TestIPtoInt_AllZeros: Tests the conversion of an IP address with all octets being zero.
• TestIPtoInt_AllOnes: Tests the conversion of an IP address with all octets being 255.
• TestIPtoInt_LeadingZeros: Tests the conversion of an IP address with leading zeros in theoctets.
• TestIPtoInt_RandomIP and TestIPtoInt_RandomIP2: Test the conversion of random IP addresses to ensure the function works correctly for different inputs.

With that done I delved into a refactor of the method. After a few minutes I kind of just realized, using IDE refactoring options and thinking it through, I came up with the following:

I split the IP address string into an array of octets using Split('.'). Then I used Select to convert each octet from a string to an unsigned 32-bit integer using Convert.ToUInt32. I then used Aggregate to iterate over the octets and combine them into a single 32-bit integer:

• The result parameter accumulates the result of bitwise shifting and adding the octets.
• For each octet, we left-shift the result by 8 bits (<< 8) and then add the current octet to it.
• The final result is the 32-bit integer representation of the IP address.

namespace IpConverter;

public static class IpConvert
{
    public static uint ToInt32(string ip)
    {
        return ip.Split('.')
                 .Select(octet => Convert.ToUInt32(octet))
                 .Aggregate((result, octet) => (result << 8) + octet);
    }
}

With that another problem solved and code refactored. Happy thrashing code, catch ya next installment!

AI Tooling Lagniappe!

Reference

• Github with first draft and the refactoring. Then finally, the AI lagniappe!

I get asked this on a semi-regular basis and I’m finally, after all these years, getting around to writing up what exactly a principal level engineer or principal software developer is. Here is what I’ve got for you in this post, broken down to the big topical points of conjecture of what this role entails. I write conjecture, because in all seriousness, this role is as wishy washy once you include more than a handful of companies.

The Principal Software Developer – AKA principal engineer, is a senior-level professional in the software development or engineering field. This role is typically associated with significant technical expertise, leadership responsibilities, and strategic impact on projects and organizational goals. Here are the key responsibilities and attributes of a principal developer/engineer:

Key Responsibilities

Technical Leadership

• Provide technical direction and architectural oversight for projects: Define the architectural roadmap and ensure it aligns with business goals. Develop high-level designs and guide the team through the implementation of these architectures. Continuously review and refine the architecture to incorporate new technologies and respond to changing business needs.
• Ensure that software development practices align with industry standards and best practices: Implement and enforce best practices in coding, testing, and deployment. Regularly update the team on emerging trends and standards in software development. Conduct periodic audits to ensure compliance with established practices.
• Mentor and guide less experienced developers, promoting professional growth and skills development: Organize regular training sessions and code review meetings. Provide one-on-one mentoring and career guidance. Foster an environment that encourages learning and knowledge sharing.

Project Oversight

• Lead and participate in the design, development, and implementation of complex software solutions: Take charge of the end-to-end software development lifecycle for critical projects. Engage in hands-on coding, particularly for the most challenging components. Ensure solutions are scalable, maintainable, and robust.
• Collaborate with cross-functional teams (product managers, designers, QA, operations) to ensure project success: Facilitate clear communication and coordination among different team members. Ensure that all requirements are gathered and understood by the development team. Address any blockers or issues that arise during the project.
• Handle technical project management tasks, including task estimation, prioritization, and tracking: Break down projects into manageable tasks and assign them to team members. Use project management tools to track progress and adjust priorities as needed. Ensure that projects are delivered on time and within budget.

Innovation & Problem Solving

• Identify and propose new technologies, tools, and processes to improve efficiency and quality: Continuously scout for new technologies that can provide a competitive edge. Evaluate their potential impact on current and future projects. Develop a roadmap for adopting these technologies.
• Address and solve complex technical challenges, often requiring innovative and creative approaches: Tackle the toughest problems that arise during development. Encourage team members to brainstorm and think outside the box. Implement solutions that not only solve the problem but also prevent future occurrences.
• Conduct technical research and proof-of-concept projects to evaluate potential solutions: Stay abreast of technological advancements and industry trends. Conduct experiments and proof-of-concept projects to test the viability of new solutions. Present findings and recommendations to the team and stakeholders.

Code Quality & Best Practices

• Establish and enforce coding standards, code reviews, and testing protocols: Define clear coding standards and guidelines. Implement a rigorous code review process to ensure code quality and consistency. Integrate automated testing into the development workflow.
• Ensure that the development process includes adequate documentation and knowledge sharing: Create and maintain comprehensive documentation for all projects. Encourage developers to document their code and share insights with the team. Use documentation tools and platforms to keep information organized and accessible.
• Promote a culture of continuous improvement and technical excellence: Encourage team members to seek out and implement improvements in their work. Celebrate successes and learn from failures. Invest in tools and training that help the team stay at the forefront of technology.

Stakeholder Communication

• Communicate technical concepts and project status effectively to non-technical stakeholders: Translate complex technical information into terms that non-technical stakeholders can understand. Provide regular updates on project progress and any issues that may arise. Use visuals and presentations to enhance communication.
• Provide insights and recommendations to senior management regarding technical strategy and resource allocation: Analyze project data and performance metrics to provide actionable insights. Recommend adjustments to resource allocation to optimize project outcomes. Align technical strategies with the overall business objectives.
• Represent the technical team in meetings and discussions with clients or partners: Act as the primary technical point of contact in client and partner meetings. Ensure that the technical aspects of the project are clearly communicated and understood. Address any technical concerns or questions raised by clients or partners.

Attributes and Skills

Expertise & Experience

• Extensive experience in software development, often with proficiency in multiple programming languages and frameworks: With years of hands-on development experience, an expert in this role is proficient in languages like Java, Python, C#, and JavaScript. They have a deep understanding of various frameworks such as Spring Boot, .NET, React, and Angular. Their extensive background allows them to adapt quickly to new technologies and methodologies, ensuring they can handle a wide range of development tasks.
• Deep understanding of software architecture, design patterns, and system integration: They possess a thorough knowledge of architectural styles and patterns, such as microservices, MVC, and serverless architectures. They can design and implement scalable, maintainable systems that integrate seamlessly with other applications and services. Their expertise ensures that systems are built to be robust and future-proof, capable of evolving with the business needs.

Leadership & Mentorship

• Strong leadership skills, with the ability to inspire and guide a team of developers: They lead by example, demonstrating dedication, expertise, and a strong work ethic. They create a vision for the team and motivate members to achieve their best. Through clear communication and effective delegation, they ensure that the team works cohesively towards common goals.
• Effective mentorship abilities, helping junior developers grow and succeed: They take an active role in the professional development of less experienced team members. By providing constructive feedback, conducting regular one-on-one sessions, and facilitating learning opportunities, they help junior developers improve their skills and advance in their careers. They foster a supportive environment where team members feel valued and encouraged to grow.

Problem-Solving & Innovation

• Analytical and critical thinking skills, capable of tackling complex technical issues: They excel at breaking down complex problems into manageable components. They use a systematic approach to analyze issues, identify root causes, and develop effective solutions. Their critical thinking skills enable them to foresee potential problems and proactively address them.
• Creativity and innovation in designing solutions and improving processes: They are always looking for new ways to enhance efficiency and effectiveness. They experiment with innovative technologies and methodologies to find better solutions. Their creativity leads to unique approaches that can give their team or organization a competitive edge.

Communication & Collaboration

• Excellent communication skills, both verbal and written: They can convey complex technical concepts in a clear and concise manner. Whether writing technical documentation, preparing reports, or giving presentations, they ensure their message is easily understood by their audience. Their communication skills foster better understanding and cooperation among team members and stakeholders.
• Ability to work collaboratively with diverse teams and manage stakeholder expectations: They thrive in team environments, valuing diverse perspectives and leveraging the strengths of each team member. They are adept at managing stakeholder relationships, ensuring that expectations are clearly communicated and met. Their collaborative approach helps in building strong, cohesive teams that work efficiently towards common objectives.

Adaptability & Learning

• Willingness to continuously learn and adapt to new technologies and industry trends: They are committed to lifelong learning, always seeking to expand their knowledge and skills. They stay up-to-date with the latest technological advancements and industry best practices. Their eagerness to learn ensures that they remain at the forefront of their field, capable of implementing cutting-edge solutions.
• Ability to thrive in a fast-paced, dynamic work environment: They are comfortable with change and can quickly adapt to new challenges and priorities. Their flexibility allows them to manage multiple projects and tasks simultaneously. In dynamic work environments, they maintain high performance and productivity, ensuring that objectives are met despite the fast pace.

Impact

The principal developer/engineer plays a crucial role in shaping the technical direction and success of projects and the organization as a whole. Their leadership and expertise ensure that the team delivers high-quality, scalable, and maintainable software solutions, driving innovation and achieving strategic business objectives.

Principal Level Scenarios

Now with the definition defined, expectation of skills and experience, it’s time to delve into some scenarios that a person in this principal level role would undertake.

Scenario 1: Leading a Strategic Technology Overhaul

Context: The company has been using a legacy system for years, which has become a bottleneck for scalability and maintenance. The decision has been made to migrate to a modern, cloud-native architecture.

Action: The principal engineer leads the strategic technology overhaul. They begin by conducting a thorough assessment of the current system’s limitations and requirements for the new architecture. They then design a comprehensive migration plan that includes selecting appropriate cloud services, defining the new system architecture, and establishing a phased implementation strategy to ensure minimal disruption to ongoing operations.

Impact: By providing clear technical direction and oversight, the principal engineer ensures the successful migration to the new architecture, significantly improving system performance, scalability, and maintainability. This sets a precedent for how major technological transitions should be handled within the organization.

Scenario 2: Mentoring and Up-skilling the Team on Advanced Technologies

Context: The company aims to leverage machine learning (ML) to enhance its product offerings, but the development team lacks expertise in this area.

Action: The principal engineer, who has experience in ML, initiates a mentorship and up-skilling program. They organize a series of technical workshops and hands-on training sessions focused on ML concepts, tools, and frameworks relevant to the company’s goals. The principal engineer also works on a pilot project to demonstrate the practical application of ML, involving the team members throughout the project to provide real-world experience.

Impact: The team’s newfound skills in ML open up new avenues for innovation within the company, leading to the development of advanced, data-driven features. The principal engineer’s role in this initiative not only accelerates the team’s learning curve but also establishes a culture of continuous learning and adaptation to new technologies.

Scenario 3: Innovating Tooling to Improve Developer Productivity

Context: Developers in the organization spend a significant amount of time managing dependencies and setting up local development environments, which slows down the development process and increases onboarding time for new hires.

Action: The principal engineer identifies the need for a more streamlined approach to managing development environments and dependencies. They lead the initiative to implement containerization using Docker and orchestrate the environments with Kubernetes. The principal engineer designs a comprehensive setup that includes pre-configured Docker images and scripts, ensuring that developers can quickly spin up their environments with minimal manual intervention.

Impact: The new tooling dramatically reduces setup time, allowing developers to focus more on coding and less on environment configuration. It also ensures consistency across different development environments, reducing the “it works on my machine” issues. This innovation not only boosts productivity but also sets a precedent for adopting modern development practices.

Scenario 4: Emphasizing the Key Value of Engineering Teams

Context: The company is undergoing rapid growth, and there’s a need to highlight the strategic importance of the engineering team to senior management and other departments.

Action: The principal engineer takes the lead in showcasing the engineering team’s value. They prepare a detailed presentation that highlights the team’s achievements, such as successful project deliveries, innovative solutions, and contributions to the company’s bottom line. The principal engineer also outlines future initiatives and how they align with the company’s strategic goals. They emphasize the importance of ongoing investment in the team’s skills and technology stack to maintain a competitive edge.

Impact: Senior management gains a deeper appreciation for the engineering team’s contributions, leading to increased support and resources for future projects. This effort by the principal engineer not only strengthens the team’s position within the organization but also sets a benchmark for how the value of engineering should be communicated and recognized.

Scenario 5: Leading an Initiative to Improve Code Quality and Maintainability

Context: The codebase has grown significantly over the years, leading to increasing technical debt and making it harder to maintain and extend the software.

Action: The principal engineer spearheads a comprehensive code quality improvement initiative. They introduce and enforce coding standards and best practices across the team. The principal engineer implements mandatory code reviews and automated static code analysis tools to ensure adherence to these standards. They also organize regular “code cleanup days” where the team focuses on refactoring and reducing technical debt.

Impact: The overall quality and maintainability of the codebase improve significantly, leading to fewer bugs and faster development cycles. The principal engineer’s leadership in this initiative not only enhances the team’s coding practices but also instills a culture of quality and continuous improvement, setting a standard for future development efforts.

Principal In Summary

The principal engineer plays a crucial role in driving strategic initiatives and fostering a culture of excellence within the engineering team. By leading efforts such as a major technology overhaul to a cloud-native architecture, innovating tooling with containerization to streamline development environments, and implementing a robust CI/CD pipeline, the principal engineer addresses critical gaps in technical capability and operational efficiency. Additionally, by mentoring the team on advanced technologies like machine learning, showcasing the strategic value of the engineering team to senior management, and leading initiatives to improve code quality and maintainability, the principal engineer ensures that the team is well-equipped with the necessary skills and best practices. This leadership not only fills existing gaps but also strengthens the team by providing a balanced approach to technical challenges and fostering continuous professional growth, ultimately driving the team’s success and aligning it with the organization’s strategic goals.

References

• For some ideas about where a principal role would fit in the life span of an organization’s product, check out this post I wrote on “A Guide Path for Strategic Growth and Leadership in Software Development“.

Programming Problems & Solutions : “Conquering the Top Words Challenge in C#: A Tale of Regular Expressions and LINQ Magic”. The introduction to this series is here and includes all links to every post in the series. If you’d like to watch the video (see just below this), or the AI code up (it’s at the bottom of the post) they’re available! But if you just want to work through the problem keep reading, I cover most of what is in the video plus a slightly different path down below.

AI Refactoring & Work video at the bottom of the post.

Today I’m diving into a fascinating coding challenge that had me scratching my head for a bit. But fear not, for I have emerged victorious with a solution that will make your C# heart sing!

The Challenge

Imagine you’re given a string of text, filled with words, punctuation, and even the occasional line break. My mission, should I choose to accept it – obviously I will – is to write a function that returns an array of the top 3 most occurring words in descending order. Sounds simple enough, right? Well, let’s add a twist!

Here are the assumptions:

• A word is a string of letters (A to Z) optionally containing one or more apostrophes (‘) in ASCII.
• Apostrophes can appear at the start, middle, or end of a word (‘abc, abc’, ‘abc’, ab’c are all valid).
• Any other characters (e.g., #, , /, . …) are not part of a word and should be treated as whitespace.
• Matches should be case-insensitive, and the words in the result should be lowercased.
• Ties may be broken arbitrarily.
• If a text contains fewer than three unique words, then either the top-2 or top-1 words should be returned, or an empty array if a text contains no words.

The Tests Before we embark on this coding adventure, let’s take a look at some sample tests to guide our way:

[Test]
public void SampleTests()
{
    Assert.That(TopTrioWords.Top3("a a a b c c d d d d e e e e e"), Is.EqualTo(new List<string> { "e", "d", "a" }));
    Assert.That(TopTrioWords.Top3("e e e e DDD ddd DdD: ddd ddd aa aA Aa, bb cc cC e e e"), Is.EqualTo(new List<string> { "e", "ddd", "aa" }));
    Assert.That(TopTrioWords.Top3(" //wont won't won't "), Is.EqualTo(new List<string> { "won't", "wont" }));
    Assert.That(TopTrioWords.Top3(" , e .. "), Is.EqualTo(new List<string> { "e" }));
    Assert.That(TopTrioWords.Top3(" ... "), Is.EqualTo(new List<string>()));
    Assert.That(TopTrioWords.Top3(" ' "), Is.EqualTo(new List<string>()));
    Assert.That(TopTrioWords.Top3(" ''' "), Is.EqualTo(new List<string>()));
    Assert.That(TopTrioWords.Top3(
        string.Join("\n", [
            "In a village of La Mancha, the name of which I have no desire to call to",
            "mind, there lived not long since one of those gentlemen that keep a lance",
            "in the lance-rack, an old buckler, a lean hack, and a greyhound for",
            "coursing. An olla of rather more beef than mutton, a salad on most",
            "nights, scraps on Saturdays, lentils on Fridays, and a pigeon or so extra",
            "on Sundays, made away with three-quarters of his income."
        ])), Is.EqualTo(new List<string> { "a", "of", "on" }));
}

The First Draft Solution

Alright, let’s dive into the solution! I’ll use the power of regular expressions and LINQ to conquer this challenge. Here’s the first draft code:

using System.Text.RegularExpressions;

namespace TopWords;

public static class TopWords
{
    public static List<string> Top3(string s)
    {
        var cleanedText = Regex.Replace(s, "[^a-zA-Z']", " ");

        var words = cleanedText.ToLower().Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);

        var wordCounts = new Dictionary<string, int>();
        foreach (var word in words)
        {
            if (string.IsNullOrWhiteSpace(word) || Regex.IsMatch(word, "^'+$"))
            {
                continue;
            }

            if (!wordCounts.TryAdd(word, 1))
            {
                wordCounts[word]++;
            }
        }

        var topWords = wordCounts
            .OrderByDescending(x => x.Value)
            .ThenBy(x => x.Key)
            .Take(3)
            .Select(x => x.Key)
            .ToList();

        return topWords;
    }
}

Let’s break it down:

1. I start by using a regular expression to remove all non-letter characters except apostrophes from the input string. This ensures it only considers valid words according to the given assumptions.
2. I split the cleaned text into individual words using ToLower() to convert them to lowercase and Split() to separate them based on whitespace.
3. I create a dictionary called wordCounts to store the count of occurrences for each word. I iterate over each word in the words array and update the count in the dictionary accordingly. I then skip words that are null, whitespace, or consist only of apostrophes using a regular expression.
4. To get the top 3 words, I use some LINQ magic! I sort the dictionary entries in descending order based on the count of occurrences, then alphabetically in case of ties. We take the top 3 entries, select only the words (keys) from the dictionary entries, and convert the result to a list.
5. Finally, we return the list of top words and bask in the glory of our coding triumph!

And there you have it! A solution that passes all the tests and handles even the trickiest cases with aplomb. Remember, regular expressions and LINQ are your friends when it comes to text manipulation and data querying. But wait, as always there’s more! I’ve gotta refactor this and slim it up!

Refactoring

Alright, let’s see how we can streamline the code and minimize the lines while still maintaining readability and functionality. Well ok, one could argue about the readability of this one. But in spite of arguments, I delved deeper into regular expressions to see what I could do here, what I came up with is this micro-beast of code.

public static string[] Top3(string s)
{
    return Regex.Matches(s.ToLower(), @"[a-z']+")
        .Cast<Match>()
        .Select(m => m.Value)
        .Where(word => !Regex.IsMatch(word, "^'+$"))
        .GroupBy(word => word)
        .OrderByDescending(g => g.Count())
        .ThenBy(g => g.Key)
        .Take(3)
        .Select(g => g.Key)
        .ToArray();
}

Let’s break down the changes:

1. I use Regex.Matches() to find all the valid words in the input string directly, instead of cleaning the string and splitting it into words separately. The regular expression [a-z']+ matches one or more lowercase letters or apostrophes.
2. I use Cast<Match>() to cast the MatchCollection returned by Regex.Matches() to an IEnumerable<Match>, allowing us to use LINQ methods on it.
3. I use Select(m => m.Value) to extract the matched word from each Match object.
4. I use Where(word => !Regex.IsMatch(word, "^'+$")) to filter out words that consist only of apostrophes.
5. I use GroupBy(word => word) to group the words by their value, effectively counting the occurrences of each word.
6. I use OrderByDescending(g => g.Count()) to sort the groups in descending order based on the count of occurrences.
7. I use ThenBy(g => g.Key) to sort the groups alphabetically in case of ties.
8. I use Take(3) to select the top 3 groups.
9. I use Select(g => g.Key) to extract the word from each group.
10. Finally, I use ToArray() to convert the result to an array of strings.
11. I changed the method signature has been changed to return an array of strings (string[]) instead of a list (List<string>). This is just a minor adjustment to further reduce the code.

This refactored code achieves the same functionality as the previous solution but in a more concise manner. It leverages the power of LINQ and regular expressions to minimize the lines of code while still handling all the necessary cases. If you’re not familiar with LINQ and regular expressions it’s an unreadable nightmare, but if you are, it’s a thing of beauty!

With that, hope you dug this challenge and the walk through. Cheers! 🍻

AI Lagniappe

References

• Github repo.