Implementing communities of practice in the Georgia Department of Transportation [Dec. 2014]

GEORGIA DOT RESEARCH PROJECT 1137
FINAL REPORT
IMPLEMENTING COMMUNITIES OF PRACTICE IN THE GEORGIA DEPARTMENT OF TRANSPORTATION
OFFICE OF RESEARCH

3GDOT Research Project No. 1137 Final Report
IMPLEMENTING COMMUNITIES OF PRACTICE IN THE GEORGIA
DEPARTMENT OF TRANSPORTATION
By Gordon Kingsley, Associate Professor School of Public Policy, Georgia Institute of Technology
Eric Boyer, Assistant Professor University of Texas at El Paso
Juan Rogers, Professor School of Public Policy, Georgia Institute of Technology
Janelle Knox-Hayes, Assistant Professor School of Public Policy, Georgia Institute of Technology
Contract with Georgia Department of Transportation
In cooperation with U.S. Department of Transportation
Federal Highway Administration
December 2014 The contents of this report reflect the views of the author(s) who is (are) responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the Georgia Department of Transportation or of the Federal Highway Administration. This report does not constitute a standard, specification, or regulation.

1.Report No.: FHWA-GA-14-1137

2. Government Accession No.: 3. Recipient's Catalog No.:

4. Title and Subtitle: Implementing Communities of Practice in the Georgia Department of Transportation

5. Report Date: December 2014
6. Performing Organization Code:

7. Author(s): Gordon Kingsley, Eric Boyer, Juan Rogers, Janelle Knox-Hayes 9. Performing Organization Name and Address: Georgia Institute of Technology 790, Atlantic Drive, Atlanta, GA 30332
12. Sponsoring Agency Name and Address: Georgia Department of Transportation Office of Research 15 Kennedy Drive Forest Park, GA 30297-2534

8. Performing Organ. Report No.:
10. Work Unit No.:
11. Contract or Grant No.: 0010836 (RP 11-37)
13. Type of Report and Period Covered: Final; April 2012 December 2014
14. Sponsoring Agency Code:

15. Supplementary Notes: Prepared in cooperation with the U.S. Department of Transportation, Federal Highway Administration. 16. Abstract:
This study explored strategies through which the Georgia Department of Transportation (GDOT) can develop communities of practice to help managers facilitate critical exchanges of knowledge, support organization learning, and ultimately achieve improvements in performance outcomes. Communities of practice (COPs) are groups of professionals working on behalf of an organization, who develop on-going informal knowledge ex0changes as a means of learning about and performing key job-related processes and skills. The study consisted of five main objectives: 1) to map the incidence of COP in GDOT, 2) to monitor patterns of knowledge flows through COPs, 3) to assess the costs and benefits of COP, 4) to develop a strategy for application of SharePoint to the development of COPs and 5) to assess the impact of employee retirements on knowledge retention. The study identified the practices and functions of four communities of practice within GDOT. Strategies for supporting COPs through training and the creation of SharePoint sites were applied to two of the COPs. The effectiveness of these strategies was tested by comparing the two COPs who received interventions with the two COPs with no interventions. The findings indicate that investing in communities of practice can lead to improved performance and operational efficiency. This finding was most pronounced in decentralized communities of practice where personnel are spread widely across many offices and locations within GDOT. Much of the role of supporting COPs, we find, occurs through providing increased authoritative support, improved definition of roles, and adaptation of COP knowledge to job-related skills.

17. Key Words: Communities of Practice, Knowledge Management, Knowledge Retention Strategy

19. Security Classification (of this report):
Unclassified

20. Security Classification (of this page):
Unclassified

18. Distribution Statement:

21. Number of Pages: 133

22. Price:

Table of Contents
List of Tables.....................................................................................................................................v List of Figures ................................................................................................................................ viii Executive Summary ......................................................................................................................... ix Acknowledgements .......................................................................................................................... x Introduction .....................................................................................................................................1
What the Research Says about Supporting Communities of Practice ........................................ 6 Research Plan ............................................................................................................................ 10 Task 1: Creating Demonstration Sites for Communities of Practice ..............................................13 1.1 Identifying the Communities of Practice ............................................................................. 13 1.2 Diagnosing the Existing Practices and Processes of Knowledge Sharing With all Four of the Identified Communities of Practice........................................................................................... 18
1.2.1 Knowledge Needs Identification (Baseline Survey)...................................................... 20 1.2.2 Knowledge Base in the Communities of Practice......................................................... 22 1.2.3 Personal Interest in Community Contributions............................................................ 23 1.2.4 Identifying the Boundaries of the Community and Validation Processes.................... 24 1.2.5 Knowledge Sharing Practices within the Communities of Practice.............................. 28 Summary.................................................................................................................................... 31
Task 2: Monitoring Changes in Performance, Transaction Costs, and Productivity from the Introduction of Communities of Practice.......................................................................................32
2.1 Establishing a Baseline of Transaction Costs for Sharing Knowledge ................................. 33 i

2.1.1 Knowledge Gaps between Headquarters and District Offices ..................................... 35 2.2 Implementing Interventions for the Communities of Practice ........................................... 37
2.2.1 SharePoint Site Development ...................................................................................... 37 2.2.2 Workshops for RAID and PDT COPs.............................................................................. 40 2.2.3 Monitoring of the SharePoint Sites .............................................................................. 43 2.3 Evaluating the Impacts of the Interventions on RAID and PDT........................................... 44 2.3.1 Perceived Expertise ...................................................................................................... 45 2.3.2 Consistent Changes across the Four Communities of Practice .................................... 48 2.3.3 Improving Knowledge Sharing...................................................................................... 51 2.3.4 Evidence of Improvements in Knowledge Sharing....................................................... 54 2.3.5 Performance Outcomes Unique to Centralized COPs .................................................. 57 2.3.6 Performance Outcomes Contrary to Expectations....................................................... 62 2.3.7 Preference for Interpersonal Knowledge Sharing to Support the COP........................ 66 2.3.8 Survey Data Analysis Summary .................................................................................... 66 2.4 Researcher Observations from Workshops and Personal Meetings................................... 68 2.5 Recommendations............................................................................................................... 70 Task 3: Communities of Practice as a Strategy for Mitigating Knowledge Losses from Retirements ........................................................................................................................................................ 74 3.1 Analysis of interview data ................................................................................................... 75 3.1.1 Knowledge Source (Repositories: People, Technology and/or Processes) .................. 75 3.1.2. Knowledge Type (Tacit/Explicit; Complex/Simple)...................................................... 77
ii

3.1.3 Transfer Process (Hierarchy or no hierarchy)............................................................... 78 3.1.4. Frequency and Types of Communication (Processes of validation, if any)................. 80 3.1.5. Effects and Consequences (Processes or methods of retaining the knowledge) ....... 83 3.2 Recommendations............................................................................................................... 86 3.2.1 Level of expertise of retirees ........................................................................................ 86 3.2.2 Size of social networks ................................................................................................. 87 3.2.3 Type, frequency and location of knowledge exchanges: technology, people, or documents............................................................................................................................. 88 3.2.4 Knowledge Retention Capacity .................................................................................... 90 3.2.5 Overall Retirement Recommendations ........................................................................ 91 Conclusions and Recommendations ..............................................................................................93 4.1 Patterns of Knowledge Exchanges and Conditions for COP Development ..................... 93 4.2 Performance Improvements from COP Support Interventions ...................................... 95 4.3 Retaining Knowledge from Agency Experts Nearing Retirement.................................... 98 4.4 Concluding Remark........................................................................................................ 100 References .................................................................................................................................... 101 Appendices: Research Protocols ..................................................................................................103 Appendix A. Survey 1: Baseline ............................................................................................... 103 Appendix B. Survey 2: Transaction Costs ................................................................................ 105 Appendix C. Survey 3: Post-Test.............................................................................................. 108 Appendix D. Semi-Structured Interview Protocol for Near Retirees....................................... 114
iii

Appendix E. Workshop Materials ............................................................................................ 117 Workshop #1: Knowledge Management Fundamentals......................................................... 117 Workshop #2: Fundamentals of SharePoint Usage in a Community of Practice .................... 118 Workshop #3: Knowledge Retention Fundamentals............................................................... 119 Appendix F. Supplementary Material ..........................................................................................121 Definitions of Potential COP Activities (from Table 13) .......................................................... 121
iv

List of Tables
Table 1: Attributes of Communities of Practice Table 2: Research Tasks Table 3: Community of Practice Selection Table 4: COP Membership Over Time Table 5: Data Analyzed for Task 1 (shaded) Table 6: Respondent Responsibilities and Knowledge Sharing Frequency Table 7: Respondent Expertise Table 8: Interest in the Community Table 9: The Breadth of Knowledge Exchanges within GDOT Table 10: Reliance on Knowledge Sources Outside of GDOT Table 11: Knowledge Validation Sources for COPs Table 12: Knowledge Validation Processes Table 13: Summary of Current COP Practices (2013) Table 14: Knowledge Transaction Cost Summary Table 15: Respondent Expertise Table 16: Workshop Attendance Summary Table 17: Intended Contributions of Workshop Table 18: Workshop Evaluation Summary Table 19: Survey Response Rate Summary Table 20: Survey Data Code Summary Table 21: Comparing Expertise (Mean Score Comparison) Table 22: Comparing Expertise (Test for Significance) Table 23: Number of Persons Contacted with Questions (Mean Score Comparison) Table 24: Number of Persons Contacted with Questions (Test for Significance) Table 25: Frequency of Questions (Mean Score Comparison)

3 11-12
14 18 19 22 23 24 25 26 27 28 30 35 36 41 41 42 45 45 47 47 49
49
50
v

Table 26: Frequency of Questions (Test for Significance)

50

Table 27: Locating Written Knowledge (Mean Score Comparison)

52

Table 28: Locating Written Knowledge (Test for Significance)

52

Table 29: Locating People (Mean Score Comparison)

52

Table 30: Locating People (Test for Significance)

53

Table 31: Identifying More Accurate Knowledge (Mean Score Comparison)

53

Table 32: Identifying More Accurate Knowledge (Test for Significance)

53

Table 33: Improving Job Effectiveness (Mean Score Comparison)

55

Table 34: Improving Job Effectiveness (Test for Significance)

55

Table 35: Reduced Time in Completing Tasks (Mean Score Comparison)

56

Table 36: Reduced Time in Completing Tasks (Test for Significance)

56

Table 37: Reduced Time in Completing Tasks (Mean Score Comparison)

56

Junior Tenure, 10 years or less transportation experience

Table 38: Reduced Time in Completing Tasks (Test for Significance) Junior

57

Tenure, 10 years or less transportation experience

Table 39: Identifying Oneself with the Community of Practice (Mean Score

59

Comparison)

Table 40: Identifying Oneself with the Community of Practice (Test for

59

Significance)

Table 41: Identifying Leaders of the Community of Practice (Mean Score

60

Comparison)

Table 42: Identifying Leaders of the Community of Practice (Test for

60

Significance)

Table 43: Understanding the Goals of the Community of Practice (Mean

61

Score Comparison)

Table 44: Understanding the Goals of the Community of Practice (Test for

61

Significance)

Table 45: Solving more Complex Problems (Mean Score Comparison)

62

Table 46: Solving more Complex Problems (Test for Significance)

62

Table 47: Frequency of In-Person Meetings by COP (Mean Score Comparison)

64

vi

Table 48: Frequency of In-Person Meetings by COP (Test for Significance)

64

Table 49: COP Characteristics by COP (Mean Score Comparison)

65

Table 50: COP Characteristics by COP (Test for Significance)

65

Table 51: Perceived Values of COP Interventions (across all four COPs)

66

Table 52: Summary Findings from the Interventions

71

vii

List of Figures

Figure 1: Practical Design Training Community of Practice SharePoint Site

38

Figure 2: RAID Community of Practice SharePoint Site

39

Figure 3: SharePoint Site Usage: PDT

44

Figure 4: SharePoint Site Usage: RAID

44

viii

Executive Summary
This study explored strategies through which the Georgia Department of Transportation (GDOT) can develop communities of practice to help managers facilitate critical exchanges of knowledge, support organization learning, and ultimately achieve improvements in performance outcomes. Communities of practice (COPs) are groups of professionals working on behalf of an organization, who develop on-going informal knowledge exchanges as a means of learning about and performing key job-related processes and skills. The study consisted of five main objectives: 1) to map the incidence of COP in GDOT, 2) to monitor patterns of knowledge flows through COPs, 3) to assess the costs and benefits of COP, 4) to develop a strategy for application of SharePoint to the development of COPs and 5) to assess the impact of employee retirements on knowledge retention. The study identified the practices and functions of four communities of practice within GDOT: Environmental Services (ENV), Geographic Information Systems (GIS), Practical Design Training (PDT), and Roundabout and Alternative Intersection Design (RAID). Strategies for supporting COPs through training and the creation of SharePoint sites were applied to two of the COPs: PDT and RAID. The effectiveness of these strategies was tested by comparing the two COPs who received interventions (PDT and RAID) with the two COPs with no interventions (ENV and GIS). The findings indicate that investing in communities of practice can lead to improved performance and operational efficiency. This finding was most pronounced in decentralized communities of practice where personnel are spread widely across many offices and locations within GDOT. Much of the role of supporting COPs, we find, occurs through providing increased authoritative support, improved definition of roles, and adaptation of COP knowledge to jobrelated skills.
The study also explored the knowledge retention in the agency. Interviews were conducted with managers nearing retirement who were also associated with the various COPs observed in this study. Through this analysis, we examined changes in knowledge management and retention practices over time. The COPs have been used to enhance knowledge retention through the following practices: 1) development of person-based knowledge and expertise databases, 2) the development of formal and informal knowledge seminars, and 3) fostering and facilitating site-specific training and shadowing between near retirees and younger managers.
ix

Acknowledgements
The authors of this research gratefully acknowledge the sponsorship and support of the Office of Research of the Georgia Department of Transportation (GDOT) in cooperation with the U.S. Department of Transportation, Federal Highway Administration. This work is prepared as part of GDOT Research Project Number RP 11-37.
We would also like to acknowledge the thoughtful guidance and assistance provided by professionals within the Georgia Department of Transportation. We benefited greatly from Rick Smith's guidance throughout the research project and Andrew Heath's review of the research. The organization of the knowledge management taskforce complemented this study, and built support for knowledge-sharing activities across the agency. Doug Cogburn and Matthew Cline were also supportive in arranging Eric Boyer's workspace in GDOT. A number people were instrumental to the planning, organizing, and facilitation of activities with the four communities of practice. Special thanks go to Andrew Heath, Andy Casey, Brent Story, David Acree, Chris Rudd, Sam Woods, Scott Zehngraff, Daniel Pass, Paul DeNard, Teague Buchanan, and Gail D'Avino. All of these people provided critical support for the engagement with the four communities of practice and the responses to the surveys, the workshop attendance, the SharePoint site design, and the overall work with the communities of practice was made possible through their interest and support of this study. We would also like to thank the IT professionals who supported the project, including Mike Purdue and Jeff Roberts. Special thanks are in order to Supriya Kamatkar for her helpful guidance and feedback during the project.
The contents, findings, and recommendations from this report represent the views of the authors of this report. They do not represent the views of the Georgia Department of Transportation or the professionals with whom we worked closely during the course of this research. Any errors or omissions are the sole responsibility of the research team.
x

Introduction
This study tests whether the transfer of knowledge among personnel engaged in engineering design and preconstruction activities can be enhanced to improve the capacity of agency personnel to learn, access knowledge in a timely basis, and retain knowledge within the agency through the development of communities of practice (COPs). The research team (we) explored this question by examining four groups of engineering design and preconstruction professionals engaged in various elements of the Georgia Department of Transportation's (GDOT) Plan Development Process (PDP) over a two year time period (2012-2014). Each group exhibited some of the rudimentary characteristics of a community of practice prior to the start of the project. Two groups were: 1) exposed to additional information about the formation and management of COPs and 2) introduced to an IT resource through the construction of SharePoint sites to see if this would better stimulate knowledge transfer and knowledge management within the group. A comparison was then conducted of the development and performance of these two groups with the two groups in which no interventions were staged by the project team.
This is, in effect, a comparison of groups that have had a systematic set of interventions aimed at stimulating knowledge management with two groups that have continued to develop knowledge sharing in a more natural course. All four groups continued to demonstrate maturation of a group identity and improvements in the capacity to share knowledge and learn. However, by comparing the groups, we learn important information about strategies to stimulate and enhance the ability of groups to better foster knowledge management practices and effective knowledge transfer.
1

We observed that knowledge management becomes a critical issue when changes are made to one of the following: key management processes, the size of the professional workforce, and the overall organization design. GDOT has experienced significant changes to the organization structure over the last 10 years including: a) several reorganizations of the agency; b) significant reduction in the agency's overall workforce including retirements of key personnel from the agency; and c) updates and modifications to core processes such as the PDP. All of these factors have had impacts (some positive and some negative) upon the ways in which vital knowledge resources in the agency are stored and the processes by which agency personnel retrieve knowledge for productive purposes.
Transportation agencies have shown an increasing interest in COPs. Studies related to transportation have found COPs to be important vehicles for strengthening production networks and contributing to the health of a state transportation agency as well as facilitating the efficient use of agency assets. COPs are also useful for achieving "just in time" learning through "just right" combinations of training and technical assistance to enhance problem solving. COPs can also be a useful strategy for adapting to shortages of personnel by facilitating learning throughout the organization (Grewal & Haugstetter, 2007).
This study builds out of an earlier project entitled "Developing Strategic Systems Supporting Communities of Practice in GDOT" (GDOT Project No. RP 10-05), where we studied the incidence of current patterns of knowledge exchanges within GDOT that exhibit patterns of interaction similar to COPs. We defined COPs to be groups of professionals from different offices across GDOT who interact on an informal basis to share experiences and knowledge regarding key skills, information and procedures that they use in their work. Our definition of COPs follows that developed by Wenger, McDermott, and Snyder (2002) who describe COPs as informal groups of professionals from different offices in one or many organizations who share
2

an interest in a professional practice and exhibit a passion for improving their skills associated with that practice. A key feature of such communities is that their members consult with each other on a regular basis to learn how to improve their skills related to a professional practice. In our first study, we summarized the research literature in building the following list of attribute characteristics of COPs (see Table 1). Through interviews and surveys with GDOT managers in the prior study, we identified several classes of knowledge exchange that exhibit three or more of these properties and could be developed into a COP. We also identified the possible gains that could be achieved through the development of COPs within GDOT. Table 1: Attributes of Communities of Practice
Informal interaction between members (i.e. communication and knowledge exchanges are not required by procedures or the organization's reporting structures)
Participants are highly motivated to improve their professional capabilities in a practice
Participants are drawn from across the organizational chart or even across organizations
Participants are aware that there is a group of individuals that share their interest in a practice
Participants share information and exchange knowledge through the group on a regular basis (at least more than once a year)
In this current project, we pursued the following objectives: 1. Examining more closely the set of existing patterns of knowledge exchanges within GDOT
and determining the conditions for their development into fully operating COPs by means of pilot demonstration sites. 2. Monitoring changes in performance, transaction costs, and productivity from the introduction of COPs.
3

3. Exploring the effectiveness of COPs as a means for coping with retirements or turnover from key personnel within GDOT. We recruited four groups from the original study to be the sites for this project.
At the outset, none of the groups self-identified as a community of practice. While there was variance in the number of characteristics of a COP that each group exhibited (see Table 1 for a list of these characteristics), each group exhibited at least three of the five characteristics. An important source of variance is that two of the more developed groups also spanned multiple levels of seniority in GDOT. Thus, for more developed groups, there was also a hierarchic element introduced into the communication flows with senior managers working along-side more mid-level and junior personnel.
The research design employs quasi-experimental elements as well as mixed methods (group observations, surveys, semi-structured interviews) allowing for the examination of the effect of a "treatment" (i.e. various interactions with the research team described below) on individual and group behavior. Two pairs of groups were examined to determine the effect of a treatment on populations with similar characteristics (Shadish, Cook, & Campbell, 2002). While two groups were designated not to experience interventions (i.e. treatments) from the research team, they do not serve as true control groups because the research team cannot (nor should attempt to) disrupt the natural conduct of work within the agency by isolating the groups from each other or prevent learning across the groups. However, the research team compared the natural development of these groups as we observed their interactions over time.
The interventions provided to two of the COPs involved 1) meetings with the leadership of the COPs throughout the term of the study, 2) developing and supporting SharePoint sites for the two COPs, and 3) conducting a 3-part workshop series on techniques for developing COPs. Our strategy of combining an IT intervention with workshops and leadership engagement builds upon recommendations from the Federal Highway Administration's efforts to develop COPs
4

across transportation agencies through an application of SharePoint (Winsor, Adams, McNeil, & Ramasubramanian, 2004).
The two COPs selected for treatments involved one that is more developed (in terms of the attributes of a COP) and one that is not. The comparison COP groups that did not receive support were selected to match the level of development of the treatment group. Both of the more developed groups had communication patterns that spanned the hierarchy of the agency.
Two types of comparisons are made of the four COPs over the two-year study. First, we examined to what extent the interventions affected the treatment groups more than the nontreatment groups. Second, we examined the two treatment groups to see if the treatments impacted the more developed, hierarchic group differently than the decentralized, less hierarchic group.
Our work plan for this project consisted of three key tasks that correspond with the objectives for the project. Task 1 was to work closely with GDOT managers engaged in existing knowledge exchanges related to the Plan Development Process and to develop these exchanges toward functioning COPs. Task 2 was to monitor existing patterns of transaction costs for individuals participating in the COP and to examine whether the cultivation of community led to improvements in accessing information. In order to complete Task 2, we reviewed the current level of efficiency of existing knowledge exchanges as a baseline. We also explored the relationship between the knowledge exchanges and existing GDOT business functions. This provided a baseline for examining the role of formal and informal processes in the exchange of knowledge. Task 3 examined whether or not COPs can be an effective tool for mitigating knowledge loss through the retirement of key personnel from GDOT.
5

What the Research Says about Supporting Communities of Practice
Public agencies have experienced a growth in interest in supporting COPs (Snyder, Wenger, & de Sousa Briggs, 2004), yet there is little understanding how to identify and cultivate these groups within government organizations. The graying of the public sector workforce, also termed the Silver Tsunami (DeLong, 2004; Elliott, 1995), has increased attention to COPs and more and more government agencies are experimenting with ways to incorporate COPs into agency functions. Past practice has been to use COPs as a means for storing critical organizational knowledge, much in the way that standard operating procedures or related documentation had been used in the past (DeLong & Davenport, 2003). The key in all cases is identifying groups of professionals organized around a critical knowledge area, who can cultivate, share, and store unwritten, tacit knowledge through their ongoing work with one another.
Much of the knowledge of how to cultivate and leverage COPs has been conducted in the private sector, as more and more companies recognize the critical importance of identifying and sharing unwritten (tacit) knowledge particularly, in knowledge-intensive industries like engineering design, chemical testing, or roadway construction (Egbu & Robinson, 2005; Nobbs, 1993; Robinson, Carrillo, Anumba, & Al-Ghassani, 2001; Sheehan, Poole, Lyttle, & Egbu, 2005; Tan et al., 2006). Highly technical companies like Dupont, for example, develop COPs around skilled technicians in order to identify ongoing problems that arise from day-to-day work to prevent larger hazards that could occur from chemical testing (Sole & Wilson, 2002). The key to Dupont's approach is the regularity of interpersonal meetings, which allows for the identification, sharing and capture of lessons that arise from daily experience.
The Chrysler Corporation adopted COPs in order to develop a more integrated approach to vehicle production. Their COPs also meet regularly, but instead of mobilizing around similar
6

technical areas (as in Dupont), they include technicians from different stages of a single vehicle's production process, in order to build a more collective, holistic understanding of the different stages of production (Wenger et al., 2002). The key to Chrysler's COP program is also interpersonal, in-person meetings, but the regularity is not as important as the diversification of the technicians who attend. In the government, federal agencies like the National Aeronautics and Space Administration (NASA) or the Department of Energy (DOE) develop COPs to mitigate catastrophes by identifying ongoing problems, and sharing lessons of how to prevent errors on high-risk projects (Rubenstein-Montano et al., 2001). The approach adopted by NASA or DOE is similar to Dupont in that they work in high-risk environments requiring the ongoing identification of potential problems in order to prevent larger catastrophes.
There are a few different ways that leaders in private and public organizations have gone about supporting COPs. One approach is to develop formal processes that necessitate interpersonal knowledge sharing through ongoing meetings and knowledge exchanges. This can involve regular events like project debriefings, weekly staff meetings, training sessions, scenario planning, or simply social events like luncheons or barbeques where professionals of similar technical backgrounds discuss day-to-day challenges and brainstorm approaches for resolving them. These formal approaches are based around the regularity of meetings, since much of the knowledge that people discover in their work is soon forgotten if there is not a means for capturing it. The ongoing meetings of this kind are some of the best ways for developing knowledge from experience particularly when a group of professionals is working in a dynamic environment where knowledge needs are constantly changing.
The second approach for supporting COPs is to create avenues for the sharing of undocumented, tacit knowledge (Hildreth & Kimble, 2004). By definition, tacit knowledge is the knowledge that people know, but which is not written down (Brown & Duguid, 2001; Cheng,
7

Wang, & Wang, 2009; Polanyi, 1967). Consider the ways that a design engineer knows how to find the best peer reviewer for a roundabout design; he or she may know who to talk to, based on his or her personal relationships, but rarely does the design engineer have a manual that guides who to choose. The only way to go about sharing such tacit knowledge is through interpersonal communication (Brown & Duguid, 1991; Sole & Wilson, 2002; Tyler, 2007).
Studies of engineering practices in the construction industry find that up to 80% of the knowledge that professionals rely upon in their work is tacit (Robinson et al., 2001) even in industries with highly developed rules and procedures for guiding technical projects. Similarly, a German design firm found that their design engineers spent more than 85% of their time working on their own, but nearly 90% of the "critical knowledge situations" of their work occurred when they interacted with their colleagues (Badke-Schaub & Frankenberger, 1999). Formal activities of knowledge sharing, like the project debriefings mentioned above, can support tacit knowledge sharing of this kind, but activities with smaller numbers of people are also effective. Additional techniques for sharing tacit knowledge can include one-on-one job shadowing or mentoring, cross-training assignments, or any activity that challenges professionals to troubleshoot problem with persons they would not interact with otherwise.
A third approach is to develop a technology-based platform to support knowledge sharing across a COP. SharePoint is one platform that could serve this function, but many other forms of software can work as well (Wenger et al., 2002). The key for technology support in a COP is options for communicating with other professionals, for identifying persons of other technical backgrounds, and for developing a shared source of documentation. Buckman laboratories, a chemical testing company based in Tennessee, adopted technology to support COPs that were spread across staff in 21 countries (Pan & Leidner, 2003). They developed
8

software for supporting chat rooms, creating databases of shared documents and guidelines, and developed lists of employee experts.
The most active of technology-supported COPs are those with active "chat rooms" or "discussion boards" where employees can pose and field questions from one another. In all environments where discussion boards of this kind are developed, it can take months or even years to generate an active online community where questions are regularly asked and answered. Techniques that have supported information exchanges on online chat rooms have included 1) developing a "code of ethics" that explains that employees will not be reprimanded for asking questions related to their jobs (showing that there are things they don't know), and that responses that are not completely accurate will not be punished, 2) cultivating a culture that explains that "anything is discussable and anyone can participate" to welcome solicitations, 3) achieving support by management for the practices, and 4) establishing formal roles and responsibilities for content experts in the community to identify and store critical documents (serving as a librarian), and to facilitate ongoing communication on discussion boards.
Past research reveals that the primary obstacle to the development of COPs is not whether or not the appropriate software has been selected or whether or not a specific knowledge-sharing activity is selected for the group to use (choosing between project debriefings, or job-shadowing, for example). The main challenge to developing, sharing and storing knowledge in COPs is cultural it's the social factors that encourage or discourage professionals to show up to meetings, to log onto chat forums, or to share knowledge with their peers. People cannot be forced to learn or to share what they know. The key is developing that interest, and developing a reason for contributing to COPs as part of a person's ongoing responsibilities. Consider the key obstructions to knowledge management identified by an NCHRP study of all fifty state DOTs in the US. The study identified the following challenges
9

associated with knowledge management programs and COP development across state
transportation departments (Ward, 2007):
Difference between management statements and actions; An enduring notion that "knowledge is power"; Apathy about sharing knowledge; "Not invented here" syndrome; Reward systems that mitigate again knowledge sharing; Differing cultures and subcultures; Knowledge travels by means of language, and there may not be a common organizational language; Considering the organization to be "machine-like"; Organizational "amnesia"; Growth in "virtual" working can hinder as well as help; Overemphasis on technology or inadequate supporting technology; and Knowledge does not grow forever, and organizations and individuals should unlearn and leave behind old ways of thinking.
The literature suggests that organizational and professional cultures pose the primary
obstacles to COP development. Potential steps for overcoming these barriers could include:
sanctioning development of ongoing events designed to share knowledge, the creation of
interpersonal knowledge-sharing activities with the explicit goal of sharing tacit knowledge, and
the development of a technology backbone to the community's development to support
knowledge sharing across geographical boundaries.
Research Plan
The prior research informed the structure and implementation of the research activities
for this study. As indicated in the proposal for this study, the research consisted of three tasks:
1) Creating Demonstration Sites for COP, 2) Monitoring the Development of COP, 3) Mitigating
Knowledge Losses from Retirements. The outline of these three tasks is summarized in the
following table (from the research proposal):
10

Table 2: Research Tasks

Tasks Task 1: Creating Demonstration Sites for COP
Tasks Task 2: Monitoring the Development of COP

Research Activities - Identify selected knowledge exchanges to develop into communities of practice - Conduct workshops with potential COP members to identify knowledge architecture of the community (i.e. key sources, stability of knowledge, accessibility of knowledge, periodicity of knowledge needs, shelf life of knowledge) - Establish necessary conditions for transitioning from knowledge exchanges to COP - Develop processes for knowledge needs identification through site development surveys, workshops, and interviews. - Develop processes for knowledge validation through site development and comparisons with strategies in external COP - Develop process for knowledge storage and retrieval designed to improve upon existing practices. - Develop comparisons when possible with strategies employed by sources external to GDOT.
Research Activities -Observe selected knowledge exchanges to establish a baseline of transaction costs associated with existing communication patterns in terms of time, effort and effectiveness. - Monitor interventions associated with the development of COP - Identify knowledge gaps between headquarters and the district offices.

Outputs - Workshops for developing COP in selected knowledge exchanges - Analysis of knowledge exchanges and their relation to COP potential - On-going operation of selected COP to serve as demonstration sites - SharePoint sites associated with each of the COP that include knowledge hub for organizing, documenting, storing, and retrieving knowledge.

Deliverables -Summary of case studies describing the development of the COP and the factors contributing to the outputs from demonstration sites as part of the draft and final report. - Operational Communities of Practice in selected knowledge exchanges to serve as demonstration sites

Outputs -Evaluation of performance associated with each site by comparing costs and benefits for knowledge exchanges before, during, and after the development of the COP.

Deliverables -Draft and final report on transaction costs associated with COP -Database on transaction costs associated with knowledge exchanges

11

Task 3: Mitigating Knowledge Losses from Retirement

-Observe selected knowledge exchanges to monitor changes in transaction costs associated with accessing and using knowledge through a COP. - Identify the patterns that lead to the effectiveness of COP. -Identify individuals who are likely to retire within the two year window of this study. Explore the types of knowledge exchanges in which they participate. - Organize the knowledge exchanges of individuals who are likely to retire into two groups: Those exchanges that are related to exchanges that have been targeted for development into a COP and those exchanges that are not related to a COP. - Establish baseline of communications associated with all knowledge exchanges. - Monitor disruption that retirement brings to knowledge exchanges comparing and contrasting those exchanges that are part of a COP and those that are not.

-Data on changes in transaction costs for knowledge exchanges due to retirements. - Workshops with COP on strategies to cope with retirements.

- Draft and final report on the impact of retirement on transaction costs of knowledge exchanges comparing COP and non-COP responses. - Database on transactions costs and knowledge exchanges

To complete the tasks identified in the table, data were collected throughout the study. The data were primarily collected through online surveys administered to the COPs. However, a three-part workshop series was also conducted for the two treatment groups, offering opportunities for gathering additional feedback from members of the COPs. A round of semistructured interviews was also conducted with near retirees to complete Task 3 of the study. Regular meetings with the leadership and members of the four COPs were conducted throughout the study. Also the patterns of use for the two SharePoint sites were monitored throughout the life of the project.

12

Task 1: Creating Demonstration Sites for Communities of Practice
Task 1 was implemented around three major aims: 1) identifying the COPs to be examined and supported in this study, 2) diagnosing the existing practices and processes of knowledge sharing with all four of the identified COPs and 3) developing a number of interventions to support the development of each COP. This stage of the study considered data from three forms of data collection as described in Table 5.
1.1 Identifying the Communities of Practice
We examined the clusters of knowledge exchanges identified during the first project and selected those clusters that exhibited the highest number of attributes associated with a COP. This information provided a starting point for identifying our demonstration sites. Our criteria for selection for the groups included the following: 1) that the group exhibited three or more characteristics of a community of practice; 2) that the group was sanctioned by GDOT management as serving a productive purpose for the agency; 3) that members of the group (particularly the leadership of the group) were willing to participate over the time period of this study; and 4) that members viewed the group as a vehicle through which knowledge associated with key work processes could be acquired, exchanged, and retained.
In the previous study, two additional factors were important in shaping exchanges of knowledge. First, some of the exchanges were amongst groups of managers that spanned multiple levels of management within GDOT. This introduced an element of hierarchic influence into the knowledge exchanges. Second, some of the exchanges spanned a wider geographic range of GDOT offices across the state. Both factors were considered in the selection of the groups for this study as we selected two COPs from a cluster exhibiting varying degrees of
13

hierarchic exchanges as well as two COPs from a cluster exhibiting varying degrees of

centralization (i.e. spread across divisions and offices in the agency).

Table 3: Community of Practice Selection

Non Treatment Groups

COP: Lower Levels of Hierarchy and Decentralized Exchanges
Geographic Information Systems (GIS)

COP: Greater Levels of Hierarchy and
Centralized Exchanges
Environmental Services (ENV)

Treatment Groups

Roundabout and Alternative Practical Design Training

Intersection Design (RAID)

(PDT)

As indicated in the table above, the two COPs that received interventions in this study were the COPs for Roundabouts and Alternative Intersection Designs (RAID) and the COP organized around Practical Design Training (PDT). RAID demonstrated a more decentralized COP and PDT demonstrated a more centralized COP in the study. Geographic Information Systems (GIS) was also identified as a decentralized COP, and Environmental Services (ENV) as a centralized COP. GIS and ENV were examined in this study, but did not receive support from the research team in developing the COPs. All four COPs also meet the criteria of operating informally and spanning multiple offices and divisions across the agency. The PDT COP, for example, emerged informally as managers in roadway design identified the need for better identification and preservation of core techniques in practical design. RAID and GIS illustrate some of the most widely dispersed COPs in the study, as each one includes participants from multiple district offices.
The COP for Environmental Services (ENV) developed as a result of professionals working in the Environmental Services Office of GDOT, where they constantly deal with changes in federal interpretations of environmental guidelines. The Environmental Services division
14

itself is divided among functional divisions, addressing issues as diverse as archeology, history, air and noise issues, ecology, and regulations associated with the National Environmental Protection Act (NEPA). There has been a long-established tradition of meeting informally with professionals across the COP, where professionals often "walk across the hall" to brainstorm with one another about how to deal with emerging problems and issues. An interesting dimension of this practice is that it is boundary spanning, and involves coordination across the functional divisions of the ENV office. The COP essentially emerged in response to the need to better understand federal requirements for environmental regulations. The factors that contribute to COP's functions include support from supervisors in the environmental services division in the agency, and the co-location of members of the COP within the headquarters of GDOT.
The COP for GIS professionals (GIS) developed as a result of GDOT's Information Technology (IT) Office responding to questions from professionals, who work with variations of mapping technology, across the agency. A SharePoint site was created to post solutions, and a listserv was developed to post information to the members at large. The members of this COP generally include people who had a mapping technology downloaded on their GDOT computers. The members display varying levels of expertise in this technology, and some members use the technology more than others. This group is widely distributed across the organization. Factors that contribute to the COP's functions include personnel who were hired to coordinate trainings and a SharePoint site for GODT professionals working on GIS and related mapping technology in the agency. The two persons hired for these coordinating roles began their work in this area over the past year when the study was conducted.
The COP for professionals working on practical design training issues developed as a result of GDOT's recognized need to cultivate expertise in core areas of engineering design for
15

preconstruction activities. Six technical areas were identified for internal training, and instructors were selected by leadership to develop curriculum and to train their colleagues. Over time, new instructors were selected and developed for trainings. The members of the COP are generally located in GDOT's headquarters and use skills from the trainings in their everyday work. Factors that contribute to the COP's functions include leadership support for the trainings and the close proximity of members of the COP to one another.
The COP for professionals working on Roundabouts and Alternative Intersection Designs (RAID) developed as a result of increased interest in roundabouts in GDOT. Projects in these areas emerge throughout the state, and the development of the COP was an effort by some of the leading experts in the COP to share experiences and to streamline approaches to roundabout development in the agency. A listserv was created for members of the RAID team, and knowledge exchanges were organized to share experiences among designers working on roundabouts from across the state. Factors that contribute to the COP's functions include an interest in projects of this kind among the group's members, the group's listserv and the efforts of the Office of Traffic Operations to organize documents and resources for the RAID community.
In each pilot demonstration, the research team traced the arc of development from current forms of knowledge exchange to a more fully developed COP. The demonstration sites were necessarily smaller in scale than the actual community of potential participants simply due to the fact that not all GDOT personnel may be able to participate in the research. In other words, not all members of all four COPs were able to contribute to the surveys in the study, to attend the workshops, and/or participate in the development of the SharePoint sites. This works to the advantage of the research project. Each pilot demonstration began with a nucleus of willing participants to define the knowledge content, develop validation strategies, and
16

develop strategies for facilitating knowledge exchanges. The number of persons identified in
each of the COPs is outlined in Table 4.
After this initial phase, GDOT participants invited colleagues who were unable to
participate in the initial discussions to review and use the resources developed to date. They
heard from their colleagues about the development of the COP and the ways in which it could
be used. Feedback from potential participants was used to adapt knowledge content,
validation, and exchange processes for the COP. We then repeated this process of COP
development and review by potential participants throughout the life of the project. In this
way, we intended to develop the robustness of knowledge resources and diffuse awareness of
the COP.
The strategy for developing each pilot demonstration into a COP was contingent upon
the current state of knowledge exchange practices. The research team worked with participants
in each pilot site and recommended a strategy of development. We hypothesized that there
would not be a single set of best practices for developing COP. Rather the literature (both
research and professional) suggests that several factors will shape the appropriate strategies for
transitioning existing patterns of knowledge exchange into a COP. These factors include:
The type of knowledge being exchanged; in particular whether the knowledge content is focused on technical information, procedural information, other classes of information, or a combination of types.
The geographic distribution of the participants in the knowledge exchange. The source of the demand for knowledge; i.e. the knowledge needed to satisfy GDOT
rules and/or procedures or the demands of an external actor. The time sensitivity of the knowledge being exchanged. The sensitivity of the knowledge content being exchanged to GDOT strategies and
norms. The media of knowledge exchange. The patterns of communication relevant to each group that require support; e.g.
meetings, information repositories, networking, among others.
17

Given the long-term time frame of the study, there were some fluctuations in the

membership of the four communities. The complete lists of COP members were identified

through assistance with the leadership of each COP. While each COP had a number of persons

who left the agency, the overall membership remained fairly constant. A breakdown of the

membership numbers reported for the study is outlined in Table 4.

Table 4: COP Membership Over Time
GIS
Membership at Study 398 Inception (January 2013) Membership at Study 319 Conclusion (May 2014)

ENV

RAID

PDT

Number of Members

50

240

121

49

227

119

1.2 Diagnosing the Existing Practices and Processes of Knowledge
Sharing With all Four of the Identified Communities of Practice
Three forms of data collection were conducted in order to diagnose the existing
practices and processes of knowledge sharing within all of the identified COPs, as well as the
knowledge needs of the communities. The data collection techniques included:
An online survey measuring the "baseline" costs of sharing knowledge with the four COPs;
An online survey measuring the "transaction costs" associated with knowledge sharing among the four communities;
A three-part workshop series conducted with members of the two COPs identified for treatments in the study (PDT and RAID);
Because the aim was to diagnose the COPs, we focus on these early stages of data collection. A
breakdown of the data collection techniques and responses for the analysis of Task 1 are
identified in Table 5, as indicated by shaded rows.

18

Table 5: Data Analyzed for Task 1 (shaded)

Date January 2013 June 2013 August 2013
December 2013
February 2014
April May 2014
May 2014

Data Collection Type
Survey 1: Baseline Survey/Interviews Survey 2: Survey of Transaction Costs Workshop #1: Knowledge Management Fundamentals Workshop #2: Fundamentals of SharePoint Site Usage Workshop #3: Knowledge Management Principles and Knowledge Retention Techniques Interviews: SemiStructured Interviews with Near Retirees Survey 3: Survey of Knowledge Costs and Improvements

GIS 105/398
(54%) 22/329
(7%)
2 persons 98/319 (31%)

ENV

RAID

PDT

Number of Responses

9/50 106/240 65/121

(18%)

(44%)

(54%)

12/36 (19/240) 33/75

(33%)

(8%)

(44%)

11

17

attendees attendees

10

12

attendees attendees

27

12

attendees attendees

2

2

persons persons

34/49 (69%)

93/227 (41%)

3 persons
60/119 (50%)

The two surveys were sent to lists of COP participants identified by leaders of each of the four COPs. The baseline survey was administered through in-person interviews with the participants from the environmental COP, due to recommendations from the COP's leadership.
The workshop participants were selected based on recommendations from the leaders of the PDT and RAID COPs. There were three workshop sessions held with the PDT and RAID COPs, in order to discuss strategies related to knowledge sharing and technology use in COPs. The workshops were designed to identify the knowledge architecture of the community (including key sources, stability of accessibility of knowledge, periodicity of knowledge needs,

19

shelf life of knowledge, etc.). There were interactive activities and simulations for workshop participants to work through, in order to model and analyze knowledge-sharing practices, such as lessons-learned debriefings, and expert interviewing. Some sessions were also held with a computer set-up for each participant, in order to walk them through the SharePoint sites for each group, and to walk participants through the functions and options that the sites could offer their respective communities. A full description of the workshops and their intended contributions to the COPs is outlined in Table 16.
The workshop sessions also involved data collection, and assessments of current challenges and opportunities for knowledge sharing within the COPs. The workshop curriculum addressed the following topics: Core Concepts in Knowledge Management (Workshop 1), Strategies for Supporting Communities of Practice, Including Hands-On SharePoint Simulations (Workshop 2), and Reviews of COP Development in other DOTs, and Knowledge Retention Techniques (Workshop 3). An outline of the workshop curriculum is provided in Table 17 and a description is provided in Appendix E.
One-on-one meetings, and group meetings were also held with leaders of the four identified COPs throughout the course of the study. The meetings generally involved discussions of knowledge needs in the COPs and ongoing practices, the structures and results of the surveys, SharePoint site needs, and the structure of workshop sessions. The following sections review the findings from the surveys, workshops, and meetings mentioned above (Table 5).
1.2.1 Knowledge Needs Identification (Baseline Survey) One of the primary objectives of the first component of the study was to identify the
knowledge needs of each of the COPs. This process was critical for developing the in-person and technological support (interventions) for the COPs. In assessing the knowledge needs of the
20

communities, we explored the following issues in respect to conditions known for supporting COPs as identified in the preceding section:
the frequency of knowledge use and knowledge seeking in the COP; the relation between day-to-day knowledge needs and knowledge provided by the COP; the levels of expertise among members of the COPs. A critical question in identifying knowledge needs in a COP is how directly related is the knowledge of the community to the everyday work of the members of the community. The more directly related the knowledge needs of everyday life to the work of the community, the more we would expect that participants will engage in the community, and contribute to knowledge sharing. The stronger this relation, greater the importance of the knowledge of the COP to the immediate needs of COP members, more the opportunities that COP members have for cultivating their individual expertise.
The responses in Table 6 indicate that the Environmental Services COP (ENV) and the Practical Design Training COP (PDT) utilize knowledge that is the most directly related to the day-to-day work of the members of the COP. The two communities also involve the most frequent exchanges of knowledge among members. GIS scored the lowest on the direct applicability of the knowledge from this COP to day-to-day work, and RAID scored in between PDT (3.68) and GIS (2.28), with a score of 2.5 overall. The findings generally indicate that the members of PDT and ENV have a more direct interest and reasoning for engaging in their COP, due the relevance of the COP's knowledge to their day-to-day work.
21

Table 6: Respondent Responsibilities and Knowledge Sharing Frequency

Survey Questions (from Survey 1)

Std. Std. Error COP N Mean Deviation Mean

How often do you take responsibility for issues

related to [GIS/ENV/RAID/PDT] on transportation ENV 9

projects that you work on?

[Every project I work on =1, Most of the projects I PDT 65

work on =2, Some of the projects I work on =3, A few of the projects I work on =4, None of the

GIS 105

projects I work on =5, Other =6, No answer=0]

RAID 106

1.11
1.28 3.31 3.29

0.333
0.573 1.613 1.441

0.111
0.071 0.157 0.14

How often do you pose questions to other designers/engineers on issues related to [GIS/ENV/RAID/PDT]? [Less than once per year=1, At least once per year=2, At least a few times per year=3, At least once per month=4, At least once per week=5, No answer=0]

ENV 9 4.11 PDT 65 3.68 GIS 105 2.28

RAID 106 2.5

0.928 0.309 1.002 0.124 1.326 0.129 1.148 0.112

1.2.2 Knowledge Base in the Communities of Practice A second question was the extent of expertise among members of the community and,
in this way, the expected contributions they can add to other professionals in carrying out their work. The more expertise within the group, the more we would expect that the community is active, since there are more individuals with desired knowledge within a single community.
Members of the ENV and PDT communities also scored the highest on expertise within their respective COPs. The high level of expertise within ENV is due to the community's membership within the Office of Environmental Services, where members work on projects related to environmental regulations and approvals on a daily basis. Similarly, members of the PDT community work on transportation engineering projects on a daily basis. Interestingly, members of the RAID group report some of the lowest levels of expertise in their area. The low

22

Table 7: Respondent Expertise

Survey Questions (from Survey 1)

Std. Std. Error COP N Mean Deviation Mean

To what extent do you agree with the following statement? "I view myself as an expert on [GIS/ENV/RAID/PDT]" [1 = do not agree, 10 = completely agree]

ENV 9 PDT 65 GIS 105

8.67 6.32 4.14

2

0.667

1.751 0.217

2.966 0.289

RAID 105 3.49 2.366 0.231

score, compared to findings from related data collected, suggests that there are a small number of experts within this group who other members rely upon. The large standard deviations for these two COPs (GIS and RAID) provides further evidence of a knowledge "gap" among members of each COP, given the wider disparity in their responses. Many participants from the GIS COP indicated that they have little or no use of GIS, which could help to explain the lower levels of perceived expertise in this group.
1.2.3 Personal Interest in Community Contributions A third question was the extent that members of the COPs have a personal interest in
contributing to COPs. Put differently, to what extent do they wish to develop a virtual community and/or advise others in their community? We hypothesize that the higher the stated interest in developing the community, the greater the gains that we would expect in the COP's development during the course of the study.

23

Table 8: Interest in the Community

Survey Questions
(from Survey 1)
To what extent do you agree with the following statement? "I am strongly interested in building a virtual community* to share knowledge on [GIS/ENV/RAID/PDT] " *[A "virtual community," in this case, could refer to online discussion boards, virtual meetings through a web-based platform, or the storing of documents and postings on a platform like SharePoint.] [1=do not agree, 10=completely agree]

Std. Std. Error COP N Mean Deviation Mean

ENV 9 8.9 1.764 0.587

PDT 65 7

2.462 0.305

GIS 105 6.5 3.156 0.308

RAID 106 6.7 2.648 0.257

To what extent do you agree with the

ENV 9 5.9

following statement? "I seek

information about [GIS/ENV/RAID/PDT] PDT 65 7.5

as part of a formal process that I have to

follow in my job" [1=do not agree, 10=completely agree]

GIS 105 4.6

RAID 104 5.5

3.621 1.207 2.292 0.284 3.278 0.32 2.801 0.275

All of the COPs reported a strong interest in contributing to a virtual community, in order to share knowledge related to their areas of work. The ENV and PDT groups reported the highest scores for seeking out knowledge related to "formal" requirements within their jobs. Our data analysis indicates that within these two communities, knowledge sharing and the content of their communities are most directly tied to their day-to-day work, and the expectations of their supervisors. The hierarchical influence of their work is most directly related to the work of individuals in these two communities.
1.2.4 Identifying the Boundaries of the Community and Validation Processes The fourth question in the diagnosis examined the bounds of membership within the
COPs. Our first study (GDOT Research Project No. RP 10-05) focused on GDOT personnel

24

engaged in the Plan Development Process (PDP). We employed a snowball sampling strategy

with rounds of nominations to identify potential participants in knowledge exchanges. We

found that nominations quickly led to individuals beyond the bounds of the PDP and beyond the

bounds of GDOT. In the first study, we excluded non-GDOT participants (though we did include

non-PDP GDOT employees).

In this study, we sought to identify the natural bounds of current knowledge exchanges

and include all actors as potential participants in the COP including actors external to the PDP

process and GDOT. We examined the breadth of the COPs through a two-phased approach.

First, we examined the distribution of members of the community within the agency and the

extent that their professional roles spanned the hierarchy. Secondly, we examined the role (if

any) of external sources of knowledge for each of the COPs, to determine how directly linked

they should be with sources outside of the agency. The following calculations are derived from

mean scores of survey items from the baseline survey. The focus of the scores is on within-

agency communication and does not capture communication with other organizations.

Table 9: The Breadth of Knowledge Exchanges within GDOT

(from Survey 1)

GIS

% of exchanges within their team

65%

% of exchanges outside of

their team (across the

35%

organization)

ENV RAID PDT 100% 50% 85%
0% 50% 15%

The data indicate that GIS and RAID are the most dispersed COPs among the four under examination. Most of the members of ENV are physically located within the headquarters of GDOT, making it easier for them to communicate directly with one another on a regular basis. The scores for ENV do not capture, however, the extensive cross-divisional communication that occurs across functional divisions within the Office of Environmental Services. The same is true

25

for the PDT group. The differences are among the GIS and RAID groups. Both GIS and RAID are

generally dispersed across the headquarters and district offices of GDOT, with professionals

contributing to a shared source of knowledge who may never have seen each other before. The

dispersed nature of the RAID group was evident in the workshop sessions, for example, since

those sessions often required video-conferencing to engage professionals from district offices in

the sessions. At one session, for example, seven separate district offices contributed to the

session, adding to a group discussion that spanned eight different locations. Conversely, the

participants from PDT attended sessions in-person, since all were already located in the

headquarters building where the sessions were located. The following scores are derived from

scores on baseline surveys and research interviews, indicating mean scores for each COP.

Table 10: Reliance on Knowledge Sources Outside of GDOT

(from Survey 1)

GIS

avg. scores

Federal or state agency

Low

Transportation

associations (AASHTO, TRB,

Low

etc.)

Consulting Firms

High

ENV High Low Medium

RAID Medium Medium
High

PDT Low Medium Low

All of the COPs rely at least to some extent on outside knowledge to carry out their dayto-day work. The ENV group is the most reliant on outside information largely due to the impact of federal guidelines on the work that they do in the agency. Their reliance on federal contacts is not due to the changing nature of environmental laws per se, but due to the changing interpretations of laws by federal officials who approve their projects. The very nature of the knowledge within this COP is therefore largely dependent on feedback and input from sources outside of the agency.

26

RAID is also highly dependent on knowledge from sources outside of the agency, but

from the private sector consulting community. Roundabout plans are often "reviewed" by a

peer reviewer from a consulting firm to verify design specifications. Peer reviewers are central

to roundabout development and generally weigh-in on a project late in the project's

development. GIS professionals turn to the consulting community for technical questions, but

many of their questions are fielded by the GIS support team within the agency as well. PDT is

the most "closed" among the four COPs. While they do seek out answers from outside the

agency, they generally solve their problems by seeking solutions among themselves, or by

reviewing published design standards in the agency. Nevertheless, the PDT community must

liaise across the agency and with external agencies. Their communications reach beyond the

direct community.

Knowledge validation plays a central role in the process of identifying and acting upon

knowledge in COPs. Through meetings with professionals from the identified COPs, and the

workshops, we identified the following sources of knowledge validation.

Table 11: Knowledge Validation Sources for COPs

GIS GIS Support
Team (GDOT); External
Consultants; Guidebooks on
GIS

ENV Federal
Regulations; Supervisors in
Environmental Services; Federal Officials; State Officials

RAID

PDT

Experts in

Supervisors;

GDOT;

Subject Matter

Peer

Experts (SMEs);

Reviewers; Manuals;

Design Plans/ Training

Peer Reviews

Materials

All of the COPs rely on a combination of explicit knowledge sources (written documents), and tacit knowledge (unwritten knowledge). For the areas of GIS and PDT, a great deal of the knowledge needed is written down, either through technical guidance for GIS or through well-developed design manuals. For the ENV group, the nature of the knowledge is
27

constantly changing, placing greater emphasis on supervisory input and input from federal

authorities. For RAID, the emphasis is on precedence, and the documentation of how designs

were implemented on past projects. Because of the nature of the delivery process for

roundabouts in GDOT, a core number of members within the group review all designs for the

agency. As of the summer of 2014, these professionals are located within the Traffic Operations

Office of GDOT.

The processes of knowledge validation also reveal differing emphases on internal and

external validation sources, and differing emphases on written or unwritten guidance. The

following Table 12 captures these differences in a summary format.

Table 12: Knowledge Validation Processes

Written Sources Unwritten Sources

Sources Internal to GDOT PDT GIS

Sources External to GDOT RAID ENV

1.2.5 Knowledge Sharing Practices within the Communities of Practice

Through one-on-one meetings, workshop sessions, and survey data (as indicated in

Table 5), we identified a number of activities that the COPs currently carry out to share

knowledge within their respective communities. Breakdowns of the activities conducted at the

onset of this research project are indicated below.

Each of the communities demonstrated at least "some" level of knowledge sharing

activities, as indicated in Table 13 below. PDT is the only community with a formal training

program in place. Subject matter experts (SMEs) are identified by the leadership in the COP,

and serve as instructors in each of the six technical areas of the COP: capacity analysis and traffic

studies, drainage analysis & design, erosion control, geometric design, pavement design &

approval, signal design, and signing and marking. This list is rather unique to the PDT COP, since

they had developed their own formal training programs in these six technical areas. We are not

28

aware of any other groups in GDOT that have developed SME lists in this way, in respect to instructional roles in formal trainings. While many documents are shared within the PDT group via their R.O.A.D.S.: Repository For Online Access To Documentation And Standards website,, the materials from their trainings are not shared, and there are no lists for "graduates" of training courses to identify other members or other instructors for follow-up questions.
At the initiation of the study, the RAID group had a listserv, but had not yet experimented with SharePoint and had conducted only one in-person formal knowledge exchange over the course of the study. The GIS group had an active online presence (including listserv postings, online communication facilitated by a "coordinator," the creation of a SharePoint site, and the documentation of online resources on the SharePoint site, as indicated in Table 13), but they had few formal in-person meetings at the onset of the study. The ENV group had shared knowledge formally in their office, and conducts regular meetings to discuss changing regulations, but do little publicizing of the key federal contacts who should be consulted on changing regulations. An interesting observation is that the two more decentralized COPs (RAID and GIS) utilize a wider spectrum of knowledge sharing activities than their more centralized counterparts. The adoption of more diversified knowledge-sharing activities is likely due to absence of other formal coordination controls within the communities, due in large part to absence of formal hierarchy in the COPs.
Despite the prevalence of "some" activities to support knowledge sharing in each of the four identified communities, there was still much more that they could do to support knowledge sharing. Key activities that are missing from all groups were the use of online forums, the publicizing of SME lists, expert interviewing and the enactment of informal knowledge sharing particularly among senior and junior members of the identified COPs. DOTs, like the Alaska
29

Table 13: Summary of COP Practices as of 20131

Activities Conducted*
Formal Trainings
Formal Knowledge-
Sharing (roundtables, storytelling)
Informal Knowledge Sharing (lunches,
etc.)
Listserv Postings
Assigning a coordinator for
the COP
SharePoint Knowledge
Sharing
Online Forums
Centralizing Knowledge
Sources
Create SME lists
Expert Interviewing

GIS X
X X X X

Community of Practice

ENV

RAID

X

X

X

X

PDT X
X

*A description of the activities from this table is provided in the Appendices: Supplemental Materials, under "Definitions of Potential COP Activities"

DOT, for example, hold quarterly barbeque lunches within COPs to provide engineers with opportunities to get to know one another better and to share ideas. Similarly, the agency updates an SME "yellow pages" within COPs so that engineers know "who to go to" with questions. The Virginia DOT also coordinates regularly scheduled formal knowledge sharing activities for COPs in their agency. The COPs in VDOT are also different from those in GDOT in

1 The following source material guided the development of the columns for this table: (Lave & Wenger, 1991; Raven, 2003; Rubenstein-Montano et al., 2001; Snyder et al., 2004; Swan et al., 2002; Wenger, 2010; Wenger et al., 2002; Wenger et al., 2010; Wenger et al., 2009).
30

that that membership of VDOT's COPs is constituted by SMEs alone. Membership is more broad-based in GDOT, including members of highly variable levels of expertise. Retirees also continue to contribute to COPs in VDOT, by attending informal and formal meetings of the group.
Summary
The diagnosis of the four COPs indicates varying demands and challenges for each of the groups. The RAID and GIS groups face some of the greatest challenges in developing their communities due to the 1) cross-division structure of their memberships, 2) the diminished support of formal authority in their communities due to the absence of formal hierarchy, 3) the less direct role that the knowledge of the community plays in the work of the of the members, and 4) the lower levels of reported expertise among the members of the two COPs. We expect that the GIS and RAID COPs face the greatest challenges in improving knowledge sharing due to these structural characteristics.
Based on the activities of the four COPs identified in Table 13, we chose the following activities for supporting and developing the PDT and RAID COPs (intervention groups):
1) Develop SME lists of participants for the groups; 2) Develop SharePoint sites for the groups; 3) Conduct workshops modeling knowledge-sharing activities, and SharePoint
functionality; 4) Create centralized knowledge sources for documents and resources; 5) Model knowledge-sharing activities for the communities, including storytelling, expert
interviewing, project debriefing, and online discussions; and 6) Conduct ongoing meetings with the leadership of the COPs to determine community
needs and priorities. 31

Task 2: Monitoring Changes in Performance, Transaction Costs, and Productivity from the Introduction of Communities of Practice
Our second task was to monitor the changes in performance, transaction costs, and productivity that may occur from the maturation of the COP. The ultimate goal of any change in process was to achieve gains in performance. The chief gains from a COP can be found in the amount of time and effort that managers dedicate to acquiring and applying knowledge. At a minimum, there should be a lowering of the transaction costs associated with finding and using knowledge.
We found that the COPs were useful in all four groups for enhancing knowledge exchanges. The number of communications within and across the groups increased after the treatments were implemented. The groups that received treatment (RAID and PDT) showed the greatest benefit from intervention. However, even among these groups, the results were differentiated as physical and organizational proximity is a critical issue in the exchange of knowledge. Our analysis indicates that the PDT COP showed the greatest benefit because it is centralized; and the members identified that greater level of expertise is located at GDOT's headquarters, which further facilitates knowledge exchanges.
Finally, the examination revealed the significance of tacit knowledge to the function of the communities. The fact that knowledge exchanges increased between the periods suggests that the COP interventions and development activities helped expand communication networks within GDOT so that members could better identify the experts who can guide problem solving and process and regulation understanding. As detailed in the conclusion of Task 3, GDOT is likely to find the greatest benefit from supporting the development of tacit knowledge exchange
32

within and between the COP through a variety of recommended activities including seminars and staff exchanges.
2.1 Establishing a Baseline of Transaction Costs for Sharing Knowledge
In order to assess the performance within a COP, we administered surveys of all four COPs to establish a baseline of performance under the existing patterns of knowledge exchange. The aim of the survey was to determine the steps that professionals in each of the COPs take in order to solve knowledge-intensive challenges that they encounter in their day-to-day work. Specifically, we investigated the types of knowledge sources that professionals in each of the COPs turn to, as well as the amount of time and effort that they exert in order to solve problems they encounter.
Members of the COPs were asked to simulate an experience of solving a technical problem related to the knowledge of their respective COPs, in order to determine the types of knowledge sources sought, the number of sources that individuals consulted, on average, and the amount of time that it generally takes to solve problems related to their respective COPs. The aim was to develop an informed understanding of how much effort GDOT professionals need to exert to solve technical problems in the workplace. We asked these questions of respondents in survey 3, which, given the timing of survey 3, provides the most recent perspective on the dimensions of knowledge management issues.
The survey results illustrate many of the tangible challenges involved in accessing and utilizing critical knowledge within GDOT. One issue is whether the knowledge accessed is in an explicit form (through formal documentation or databases) or a tacit form (i.e. through another person). The extent that technical answers are solved through the identification of explicit knowledge generally represents the institutionalization of information so that expertise is preserved in the organization. In any workplace, it is difficult to document all of the knowledge
33

that guides organizational behavior. GDOT has a long history of developing procedural manuals and guiding documents to answer technical questions on transportation projects, including manuals such as the Design Policy Manual.
However, the survey data indicate that personal, tacit knowledge underlies knowledge seeking across the four COPs indicating that as valuable as documentation is (through the development of procedural manuals, guidelines, etc.) unwritten knowledge is central to day-today problem solving. When asked how many people the typical individual turns to in order to answer questions, the average number was 2 persons across the four groups.
The findings indicate that professionals consult multiple sources of knowledge in solving technical problems. On average, respondents from all four COPs consult at least three different knowledge sources in order to identify answers to the questions that they encounter in the workplace. This finding implies that understanding who to go to for questions is as important as understanding which document to turn to in developing answers to questions.
The findings also illustrate the impact on productivity for accessing the correct person or the correct document in solving challenging problems. Specifically, typical problems take 15 minutes or more to solve. The amount of time is slightly higher for professionals from the environmental COP, yet the amount of time and effort is similar across the remaining three communities. Knowledge seeking plays an important role in the day-to-day work of professionals in GDOT, and communicating across individuals is fundamental to problem solving.
34

Table 14: Knowledge Transaction Cost Summary

Survey Questions (from Survey 3)

Std. Std. Error COP N Mean Deviation Mean

ENV 29 2.1

How many people did you speak with, in

determining your answer to this question? PDT 58 1.72

[0 none, 1 person, 2 people, 3 people, 4

people, 5 more than 4 people]

GIS 76 1.5

1.081 0.201 0.586 0.077 0.872 0.1

RAID 72 1.71 0.956 0.113

Please indicate the number of knowledge "sources" you contacted in solving the technical question [Source Sum = 0, 1, 2, or 3 sources]

ENV 30 2.933 PDT 58 2.966 GIS 81 2.753

0.254 0.263 0.681

0.046 0.034 0.076

RAID 79 2.848 0.483 0.054

ENV 29 3.45

How long did it take you to come up with

an answer to this question?

PDT 58 2.98

[0=No answer, 1=5 minutes or less, 2=5-15

minutes, 3=15-30 minutes, 4=more than 30

minutes]

GIS 74 3.05

RAID 72 3.18

0.985 0.183 0.827 0.109 1.045 0.122 0.909 0.107

2.1.1 Knowledge Gaps between Headquarters and District Offices A second question involves the extent that knowledge is distributed across
headquarters and district offices involved in the COPs. Two of the COPs are concentrated within the GDOT's headquarters: the environmental COP (ENV) and the practical design training COP (PDT). For these two COPs, the findings indicate knowledge exchanges primarily between persons within offices located at GDOT headquarters. The GIS COP (GIS) and the roundabout COP (RAID), on the other hand, are widely distributed across GDOT offices. The primary source
35

of expertise for the GIS COP is the GIS Solutions Center, located in GDOT's headquarters. Many

members of GIS from across the state turn to experts in headquarters for guidance on the

program. The primary source of knowledge for RAID was divided between headquarters and

the State Traffic Operations Office for most of the timeframe of this study. With one of the

members transitioning from the Design Policy and Support Office to District 3 during the time of

this study, more attention is on the Traffic Operations Office for supporting roundabout projects

across the state. An important question is the extent that expertise is distributed and shared

across communities, like GIS and RAID, which are diffused throughout the agency.

The findings indicate that respondents are more likely to view themselves as experts in

COPs when they are concentrated in headquarters and associated with knowledge exchanges

that include hierarchic relationships in structure. These people are focused on issues most

directly related to their job descriptions. Table 15 illustrates that the ENV and PDT COPs (both

concentrated in GDOT) scored much higher in their self-reported agreement with a statement

Table 15: Respondent Expertise Survey Question

Std. Std. Error COP N Mean Deviation Mean

To what extent do you agree with the following statement? "I view myself as an expert on [ENV, GIS, PDT, RAID]" [1 = do not agree, 10 = completely agree]

ENV 9 8.67 PDT 65 6.32 GIS 105 4.14 RAID 105 3.49

2

0.667

1.751 0.217

2.966 0.289

2.366 0.231

expressing their own personal expertise in topics related to their respective COPs (ENV = 8.67/10 and PDT 6.32/10) in comparison to the COPs spread across the agency (GIS =4.14/10 and RAID 3.49/10). This difference implies a knowledge gap between headquarters and district
36

offices for at least two of the COPs, and that there are barriers to sharing expertise across divisions in GDOT.
The summary findings suggest very real challenges for transportation professionals in accessing and identifying the types of knowledge that they need to solve complicated problems in the workplace. The findings also suggest greater challenges for sharing knowledge among the COPs that are spread across the organization (GIS and RAID), due to some extent on the reliance on subject matter experts within GDOT's headquarters.
2.2 Implementing Interventions for the Communities of Practice
The interventions provided by the research team for the treatment groups involved three complementary activities: 1) meeting with leadership in the RAID and PDT COPs to identify knowledge sharing needs and to identify potential strategies for community support; 2) developing and monitoring SharePoint sites for each of the two COPs; and 3) conducting a three-part workshop series to demonstrate the functions of the SharePoint sites, to create opportunities for community building, and to model knowledge management strategies that could benefit either community.
2.2.1 SharePoint Site Development A SharePoint site was developed, in coordination with the leadership of the RAID and
PDT COPs, with the following goals in mind: 1) to create a shared source of documentation related to the respective COPs, 2) to create a space where members of the communities could interact with one another and identify potential contacts for their questions, and 3) to create an online space where questions and answers could be exchanged and documented for future use. The sites were created as "my teams" sites, and samples of how they appear are provided in Figures 1 and 2.
37

SharePoint site includes a sub-site for each of the training areas provided in practical design: capacity analysis and traffic studies, drainage analysis & design, erosion control, geometric design, pavement design & approval, and signal design. The sub-sites include instructional materials from the training sessions and lists of professionals who completed training in each of the six areas. Subject matter experts (SMEs) are also identified on the SharePoint site, as indicated the persons listed as instructors for the each of the six areas. All PDT SME's are GDOT employees who had been selected by leadership to serve as instructors. The aim of compiling the lists of professionals by training area is to improve the connection between SMEs and persons who attended their sessions, and to improve the ability of training alumni to connect with one another. Figure 1: Practical Design Training Community of Practice SharePoint Site http://gdotteams.dot.ga.gov/offices/ord/Pratical%20Design%20Training/default.aspx The PDT
38

Figure 2: RAID Community of Practice SharePoint Site http://gdotteams.dot.ga.gov/info/raid/default.aspx
The RAID site includes documents that provide guidance to professionals working on roundabouts from across the agency. Design plans, peer reviews, and related documents on specific roundabout projects are provided in a shared folder on the site. Documents such as these are otherwise stored in district offices where the projects are conducted, and a shared site provides a resource that professionals from across the agency can access more easily. Additionally, the list of RAID members is outlined on the site, so that members of the community can achieve a better understanding of who else works on these projects across the agency. The only SME's formally identified were advisers to the COP, including personnel in the Office of Design Policy and Support, and Office of Traffic Operations. Both of the SharePoint sites (PDT and RAID) provide repositories for documents, and contact information for the members of the COPs.
39

2.2.2 Workshops for RAID and PDT COPs A three-part workshop series was also implemented in order to review core concepts
related to knowledge management, to review the SharePoint sites, and to discuss potential programs in knowledge retention. An over-arching goal of the workshops was to create a forum for cultivating community among the respondents. The sessions involved simulations and interactive group activities to simulate and model activities for knowledge sharing such as structured and unstructured lessons-learned debriefings and expert interviewing. One entire session of the 3-part series was devoted to the use of the SharePoint sites and potential applications of the sites for knowledge sharing in the COPs. Descriptions of the workshops are outlined in the appendices (Appendices: Workshop Materials).
40

Table 16: Workshop Attendance Summary

RAID

PDT

August

Workshop #1: Knowledge

11

17

2013

Management Fundamentals

(4 districts)

December Workshop #2: Fundamentals of

10

12

2013

SharePoint Site Usage

(3 districts)

February Workshop #3: Knowledge

27

12

2014

Management Principles and

(6 districts)

Knowledge Retention Techniques

Total Workshop Attendees:

48

41

Each workshop was guided by the literature on COPs, in order to provide content and activities to attendees that could help them in further developing their respective COPs. A full description of each workshop's curriculum is provided in the appendices and a summary of the intended contributions is provided in the following table.

Table 17: Intended Contributions of Workshops

Workshop Title Workshop #1: Knowledge

Intended Contribution of Workshop Increased familiarity with techniques for sharing

Management Fundamentals knowledge in a COP, including examples from

other industries.

Workshop #2: Fundamentals of Increased familiarity with the functions of their

SharePoint Site Usage

SharePoint site and its potentials for their COP.

Workshop #3: Knowledge

Increased familiarity with knowledge

Management Principles and

management fundamentals to lead the COP in

Knowledge Retention

the future, and techniques in knowledge

Techniques

retention.

41

Attendees of the workshops also completed anonymous feedback forms on the perceived value of the workshops. The evaluations were administered and collected by research assistants working on the study, to ensure that the instructor of the workshops (Eric Boyer) would not influence the scoring by attendees. The scores by attendees were generally positive. While not included in the table below, workshop attendees indicated that the SharePoint material covered in the workshops was generally new to them, and provided useful applications to their work.

Table 18: Workshop Evaluation Summary

All responses were scored in the following

PDT

manner: Not effective = 1, somewhat effective = (avg. score)

2, neutral = 3, effective = 4, very effective =5

11/48*

Clarity of Explanations of Material

4.1

Organization of Presentations

4.1

Facilitation of Discussion and/or Class Activities

4.2

Overview of Core Concepts in Knowledge

4.1

Management

Overview of Core Concepts in Communities of

4

Practice

Core Techniques I can consider in my work

4.2

Understanding of Knowledge Management in

4.1

Other Organizations

*Number of responses per number of attendees to the workshops

RAID (avg. score)
17/41*
4 4.1 4 3.1
3.1
3.5 3.5

Attendees also had the opportunity to provide anonymous written comments on the
workshops. Attendees also noted the following benefits of the sessions. The following
comments illustrate anonymous qualitative feedback on the sessions.
Perceived Benefits of the Workshops (by Attendees)
"the templates provided guidance for conducting unstructured and structured debriefings"
"seeing examples of knowledge management in other companies" "communicating among our peers [in the COP]" "interaction with others to discuss KM (Knowledge Management) practices in other
companies" "thinking/discussing various current problems"
42

"the discussion between members of different design groups...breaking down the silos" "voicing information that could improve knowledge sharing within roadway design" "determined the need to have more informal discussions" "discussion with the team to get help with roundabout designs" "learning what other organizations are doing to share knowledge" "talking with peers" "learning fundamentals of communities of practice" "learning details about knowledge management" 2.2.3 Monitoring of the SharePoint Sites
To evaluate the potential of the SharePoint sites to support knowledge sharing in the RAID and PDT COPs, SharePoint analytics were employed to assess the number of "hits" to the sites by day. SharePoint site usage is demonstrated in the visuals below (Figures 3 & 4). The analytics demonstrate fluctuations in use of the sites. The highest spike in usage indicates the timing of the second workshop, where the functions of the sites were reviewed in workshop sessions. The peaks in user access to the SharePoint sites around the times of the workshops suggest that the workshops have some impact on online activity. An interesting observation is that the use of the sites increases at the times of the workshops, suggesting that in-person activities can bolster online activity in a virtual forum of this kind. However, we also note persistent low-level usage outside the times of the workshops. This indicates that GDOT personnel are accessing the site and using the site as an information resource outside the clusters associated with the workshops, yet their primary usage occurred during the timeframes when the workshops were held. This pattern of lower-level use is particularly pronounced for the RAID COP (see Figure 4).
There is evidence that the sites are continuing to be utilized by the COPs, as well. The reported usage is greater for the PDT COP than for the RAID COP, yet it is important to note that the sites are continuing to be used. More than six months since the last workshop, the sites continue to function and serve a purpose for the two communities of practice.
43

Figure 3: SharePoint Site Usage: PDT
Figure 4: SharePoint Site Usage: RAID
2.3 Evaluating the Impacts of the Interventions on RAID and PDT
A number of steps were employed to evaluate the impact of the expert meetings, SharePoint site support, workshops, and related interventions on the functions of the RAID and PDT COPs. Analysis of survey data illustrates some potential areas where members of the COPs
44

improved knowledge-sharing practices when compared against their behavior prior to the

workshops and in comparison to their peers in the COPs that did not receive support. The table

below provides a summary of survey participation by COP.

Table 19: Survey Response Rate Summary

Date January 2013
June 2013
May 2014

Data Collection Type
Survey 1: Baseline Survey/Interviews Survey 2: Survey of Transaction Costs Survey 3: Survey of Knowledge Costs and Improvements

GIS
105/398 (54%) 22/329 (7%) 98/319 (31%)

ENV

RAID

Number of Responses

9/50 106/240

(18%)

(44%)

12/36 (19/240)

(33%)

(8%)

34/49 93/227

(69%)

(41%)

PDT
65/121 (54%) 33/75 (44%) 60/119 (50%)

For the purposes of comparing data from across the three surveys in statistical analysis, the

following guide describes the labeling of the data (Table 20).

Table 20: Survey Data Code Summary

ENV.S1 ENV.S2 ENV.S3 GIS.S1 GIS.S2 GIS.S3 PDT.S1 PDT.S2 PDT.S3 RAID.S1 RAID.S2 RAID.S3

Data from Survey 1 for the ENV COP Data from Survey 2 for the ENV COP Data from Survey 3 for the ENV COP Data from Survey 1 for the GIS COP Data from Survey 2 for the GIS COP Data from Survey 3 for the GIS COP Data from Survey 1 for the PDT COP Data from Survey 2 for the PDT COP Data from Survey 3 for the PDT COP Data from Survey 1 for the RAID COP Data from Survey 2 for the RAID COP Data from Survey 3 for the RAID COP

2.3.1 Perceived Expertise

The COPs concentrated within GDOT's headquarters (ENV and PDT) generally scored

higher in terms of the self-perceptions of levels of expertise when compared to their

counterparts in the more decentralized COPs (GIS and RAID). One question is the extent that

the reported levels of expertise changed over the course of time that the members of the COPs

45

received interventions (the meetings with leadership, workshops, SharePoint site development, etc.). Increased levels of expertise could indicate greater potentials for sharing knowledge across individuals, raising the overall levels of the group. An increase in the scoring of expertise could thus indicate that the interventions support critical functions within the COPs.
The analysis indicates that perceived expertise increased for the PDT COP, remained the same for the RAID COP, and decreased for the two COPs that did not receive support: ENV and GIS (Tables 21 & 22). The overall implication is that the interventions may have had some impact on the extent of knowledge sharing and the overall increase in expertise across members of the affected communities, but that change was stronger for the more centralized COP (PDT) than for the RAID group.
As indicated in Tables 21 & 22, the average score for perceived expertise for the ENV COP dropped from 8.67/10 to 7.38/10. The scoring for GIS dropped from 4.14/10 to 3.8/10. PDT's score increased from 6.32/10 to 7.12/10 and RAID's score was nearly the same, moving from 3.49/10 to 3.44/10. The difference among these two scores is due, again, to the different surveys that were administered to the same COPs (survey one administered in January 2013 and survey three in May 2014). The high scores for the ENV and PDT teams are likely due to the concentration of professionals within the communities in the same office. Additionally, the ENV and PDT members scored the highest on the measures linking the knowledge of the COP to their day-to-day work. Many individuals in the GIS COP use the software infrequently; and a number of members of the RAID COP work on roundabouts only occasionally and some not even at all, but want to learn more about them. The rows of the following tables are shaded to indicate the responses that were compared against one another.
46

Table 21: Comparing Expertise (Mean Score Comparison)

To what extent do you agree with the following statement? "I view myself as an expert on (environmental procedures / GIS / practical design / roundabouts and/or alternative intersection designs]." [1 = do not agree, 10 = completely agree]

Std.

Error

COP

N

Mean Std. Deviation Mean

ENV.S1

9

8.67

2

0.667

ENV.S3

29

7.38

1.935

0.359

GIS.S1

105

4.14

GIS.S3

76

3.8

2.966 2.917

0.289 0.335

PDT.S1

65

6.32

1.751

0.217

PDT.S3

57

7.12

2.155

0.285

RAID.S1

105

3.49

RAID.S3

72

3.44

2.366 2.385

0.231 0.281

Table 22: Comparing Expertise (Test for Significance)

To what extent do you agree with the following statement? "I view myself as an expert on (environmental procedures / GIS / practical design / roundabouts and/or alternative intersection designs]." [1 = do not agree, 10 = completely agree]

Levene's Test for Equality of
Variances

t-test for Equality of Means

Sig. (2-

Mean

Std. Error

95% Confidence Interval of the

F

Sig.

t

df

tailed) Difference Difference

Difference

ENV.S1 ENV.S3 GIS.S1 GIS.S3 PDT.S1 PDT.S3 RAID.S1 RAID.S3

Equal variances assumed
Equal variances not assumed
Equal variances assumed
Equal variances not assumed
Equal variances assumed
Equal variances not assumed
Equal variances assumed
Equal variances not assumed

0.001 0.425 1.323 0.116

0.98 0.515 0.252 0.734

1.731 1.7
0.767 0.769 -2.26 -2.23 0.114 0.113

36

0.092

13.009 0.113

179

0.444

163.308 0.443

120

0.026

107.942 0.028

175

0.91

151.94 0.91

1.287 1.287 0.34 0.34 -0.8 -0.8 0.041 0.041

Lower Upper

0.744

-0.221 2.796

0.757

-0.349 2.923

0.444

-0.535 1.216

0.442

-0.533 1.214

0.354

-1.5 -0.099

0.359

-1.511 -0.089

0.363

-0.676 0.758

0.364

-0.677 0.76

47

2.3.2 Consistent Changes across the Four Communities of Practice Some changes were noted as consistent across the four COPs, indicating that regardless
of the interventions administered, the members of the COPs experienced a number of changes due to environmental factors affecting the entire agency. Specifically, the data indicate that people speak with more persons than they did one year ago, in order to solve problems (Tables 23 and 24). The comparisons are made, again, between data from survey 2 (administered in June 2013) and survey 3 (administered in May 2014). The differences in mean scores between these two time periods are not statistically significant, yet they indicate consistent changes worthy of attention. This could be influenced by an increased understanding of "who to turn to" for questions about their work. It could also be due to the increasing "tacit" nature of knowledge needed to solve day-to-day problems, due in part to changing regulations and guidelines. These findings imply the importance of tacit knowledge across all four COPs.
Additional analysis indicates that tacit knowledge may be particularly important for the decentralized COPs. Both GIS and RAID demonstrated an increased frequency in asking questions of other persons in their COPs (Tables 25 & 26). Members of ENV, on the other hand, indicated less frequency in asking questions of others, and PDT remained roughly the same. The findings suggest that while tacit knowledge is important for all four COPs, it is particularly important in the less centralized COPs (GIS and RAID).
The findings among the different types of groups (centralized and decentralized) also indicate that the characteristics of the groups have a significant role to play and have to be taken into consideration when designing a program to support them. In other words, a different approach to supporting COPs is needed for hierarchical groups as opposed to those with less structure.
48

Table 23: Number of Persons Contacted with Questions (Mean Score Comparison)2

How many people did you speak with, in determining your answer to this question? 0- none, 1- 1 person, 2- 2 people, 3- 3 people, 4- 4 people, 5- more than 4 people

Std.

COP

N

Mean

Deviation

Std. Error Mean

ENV.S2

11

1

1.342

0.405

ENV.S3

29

2.1

1.081

0.201

GIS.S2

22

0.64

1.329

0.283

GIS.S3

76

1.5

0.872

0.1

PDT.S2

33

0.36

0.699

0.122

PDT.S3

58

1.72

0.586

0.077

RAID.S2

20

0.6

1.046

0.234

RAID.S3

72

1.71

0.956

0.113

Table 24: Number of Persons Contacted with Questions (Test for Significance)

How many people did you speak with, in determining your answer to this question? 0= none, 1= 1 person, 2= 2 people, 3=3 people, 4= 4 people, 5=more than 4 people

Levene's Test for Equality of
Variances

t-test for Equality of Means

F

Sig.

t

Sig. (2-

Mean

df

tailed) Difference

ENV.S2 ENV.S3 GIS.S2 GIS.S3 PDT.S2 PDT.S3 RAID.S2 RAID.S3

Equal variances assumed
Equal variances not
assumed
Equal variances assumed
Equal variances not
assumed
Equal variances assumed
Equal variances not
assumed
Equal variances assumed
Equal variances not
assumed

1.922 2.588 0.305 0.52

0.174 0.111 0.582 0.473

-2.698 -2.444 -3.603 -2.874 -9.918 -9.449 -4.494 -4.268

38

0.01

15.197 0.027

96

0.001

26.442 0.008

89

0

57.557

0

90

0

28.423

0

-1.103 -1.103 -0.864 -0.864 -1.361 -1.361 -1.108 -1.108

Std. Error Difference
0.409 0.452 0.24
0.3 0.137 0.144 0.247 0.26

95% Confidence Interval of the
Difference Lower Upper
-1.931 -0.276
-2.065 -0.142
-1.339 -0.388
-1.481 -0.247
-1.633 -1.088
-1.649 -1.072
-1.598 -0.618
-1.64 -0.577

2 For this analysis, a comparison was done between responses on survey 2 and survey 3 as survey 2 (conducted early in the study) referred to topics related the number of knowledge sources needed.
49

Table 25: Frequency of Questions (Mean Score Comparison)

How often do you pose questions to other engineers/designers for

guidance on [environmental procedures / GIS / practical design

issues / roundabouts]?

Less than once per year=5, At least once per year=4, At least a few

times per year=3, At least once per month=2, At least once per

week=1, No answer=0

Std.

Std.

Error

COP

N

Mean Deviation Mean

ENV.S1

9

1.89

0.928

0.309

ENV.S3

25

3.44

0.87

0.174

GIS.S1

93

3.34

0.994

0.103

GIS.S3

84

2.07

1.095

0.12

PDT.S1

65

2.34

0.989

0.123

PDT.S3

52

2.75

1.169

0.162

RAID.S1

105

3.48

1.127

0.11

RAID.S3

80

1.88

0.986

0.11

Table 26: Frequency of Questions (Test for Significance)

How often do you pose questions to other engineers/designers for guidance on [environmental procedures / GIS / practical design issues / roundabouts]? Less than once per year=5, At least once per year=4, At least a few times per year=3, At least once per month=2, At least once per week=1, No answer=0

Levene's Test for Equality of Variances

t-test for Equality of Means

95% Confidence

Sig. (2-

Mean

Std. Error Interval of the

F

Sig.

t

df

tailed) Difference Difference

Difference

Lower Upper

Equal

variances

ENV.S1 assumed 0.173 0.68

-4.51

32

0

-1.551

0.344

-2.252 -0.851

ENV.S3

Equal variances not
assumed

-4.371 13.414 0.001

-1.551

0.355

-2.315 -0.787

Equal

variances

GIS.S1

assumed 3.772 0.054*

8.102

175

0

1.273

0.157

0.963 1.583

GIS.S3

Equal variances not
assumed

8.063 168.39

0

1.273

0.158

0.961 1.584

PDT.S1

Equal variances assumed

3.984 0.048*

-2.062

115 0.041

-0.412

0.2

-0.807 -0.016

PDT.S3

Equal variances not
assumed

-2.024 99.952 0.046

-0.412

0.203

-0.815 -0.008

Equal

variances

RAID.S1 assumed

1

0.319

10.097

183

0

1.601

0.159

1.288 1.914

RAID.S3

Equal variances not
assumed

10.282 179.51

0

1.601

0.156

1.294 1.908

[* p< .05]

50

2.3.3 Improving Knowledge Sharing The survey data analysis also indicates that members of the COPs that were supported
by the research team achieved a number of improvements in how they share knowledge. We find higher scores related to the identification of documents (Tables 27 & 28) and persons (Tables 29 & 30). These results are statistically significant and indicate that the treatments provided to the COPs led to measurable improvements in knowledge sharing.
The data also indicate that members of COPs who received treatments were able to identify more accurate knowledge than their counterparts who did not receive the support from the research team (Tables 31 & 32). This result is not statistically significant implying that the results are encouraging but not definitive.
For the purposes of comparison in the following tables, we evaluated scores of survey items between two groups: 1) the combined mean scores of the communities that received treatments (RAID and PDT) and the combined mean scores of the communities that did not receive treatments (GIS and ENV). This method of comparison allows us to compare the rate of change of similarly structured COPs that did and did not receive support from the research team during the course of the study.
The improved scores on these specific items suggest that persons in the treated COPs (PDT and RAID) have an easier time in identifying the knowledge they need to carry out their work both in terms of tacit knowledge (identifying other people), and in terms of explicit knowledge (identifying documents).
51

Table 27: Locating Written Knowledge (Mean Score Comparison)

To what extent do you agree with the following statement?

I have a sense of where to locate written sources of

knowledge related to roundabouts/practical

design/GIS/environmental procedures. [1=do not agree,

10=strongly agree]

Std.

Std.

Error

COP

N

Mean Deviation Mean

Treatment 125

7.15

Non-

Treatment 104

6

2.286 2.849

0.204 0.279

Table 28: Locating Written Knowledge (Test for Significance)

To what extent do you agree with the following statement? I have a sense of where to locate written sources of knowledge related to roundabouts/practical design/GIS/environmental procedures. [1=do not agree, 10=strongly agree]

Treatment
NonTreatment

Equal variances assumed
Equal variances
not assumed

Levene's Test for Equality of Variances

F

Sig.

10.29 0.002*

t-test for Equality of Means
t
3.395
3.328

df 227 196.163

Sig. (2-

Mean

Std. Error

tailed) Difference Difference

0.001

1.152

0.339

0.001

1.152

0.346

95% Confidence Interval of the
Difference Lower Upper
0.483 1.821
0.469 1.835
[* p< .05]

Table 29: Locating People (Mean Score Comparison)

To what extent do you agree with the following statement?

I know how to locate other people with expertise related to

roundabouts/practical design/GIS/environmental procedures

in my community of practice. [1=do not agree, 10=strongly

agree]

Std.

Std.

Error

COP

N

Mean Deviation Mean

Treatment 125

7.54

Non-

Treatment 103

6.99

2.252 2.767

0.201 0.273

52

Table 30: Locating People (Test for Significance)

To what extent do you agree with the following statement? I know how to locate other people with expertise related to roundabouts/practical design/GIS/environmental procedures in my community of practice. [1=do not agree, 10=strongly agree]

Treatment
NonTreatment

Equal variances assumed
Equal variances
not assumed

Levene's Test for Equality of
Variances

F

Sig.

4.663 0.032*

t-test for Equality of Means
t
1.642
1.61

Sig.(2-

df

tailed)

226 0.102 195.778 0.109

Mean Difference
0.546
0.546

Std. Error Difference
0.332
0.339

95% Confidence Interval of the
Difference Lower Upper
-0.109 1.201
-0.123 1.214
[* p< .05]

Table 31: Identifying More Accurate Knowledge (Mean Score Comparison)

I am able to identify more accurate knowledge related to my

work, than I was one year ago. In other words, I feel more

confident in the accuracy of the knowledge that I retrieve

than I was one year ago. [1=do not agree, 10=strongly

agree]

Std.

Std.

Error

COP

N

Mean Deviation Mean

Treatment 116

5.91

Non-

Treatment 101

5.42

2.439 2.861

0.226 0.285

Table 32: Identifying More Accurate Knowledge (Test for Significance)

I am able to identify more accurate knowledge related to my work, than I was one year ago. In other words, I feel more confident in the accuracy of the knowledge that I retrieve than I was one year ago. [1=do not agree, 10=strongly agree]

Treatment
NonTreatment

Equal variances assumed
Equal variances
not assumed

Levene's Test for Equality of
Variances

F

Sig.

3.252 0.073*

t-test for Equality of Means
t
1.36
1.345

Sig.

(2-

df

tailed)

215 0.175 197.734 0.18

Mean Difference
0.489
0.489

Std. Error Difference
0.36
0.364

95% Confidence Interval of the
Difference Lower Upper
-0.22 1.198
-0.228 1.207
[* p< .1]

53

2.3.4 Evidence of Improvements in Knowledge Sharing The data analysis implies that there can be performance implications associated with
improvements in sharing knowledge. Specifically, we note two measures where the COPs receiving treatments scored higher than their matched pairs who did not receive treatment. For example, the treated groups had some improvements in their job effectiveness (Tables 33 and 34) and a greater reduction in the time that it took them to solve problems (Tables 35 and 36). The differences in these scores however are not statistically significant. Based on this survey data alone, we can conclude that the changes are moving in the right direction but there is not a measurable enough change to indicate that performance improved.
To better understand how members of the treated communities have expedited their problem-solving practices over the course of the past year, we also consider these changes with respect to the tenure of persons within the transportation industry (Tables 37 & 38). A separate survey item asked respondents: "How many years have you worked in the transportation industry?" and we divide the responses by those that are greater than 10 years (senior tenure), and those less than 10 years (junior tenure). When considering the amount of employee tenure in respect to reductions in problem solving times over the past year, the results indicate that "junior tenure" employees achieved a statistically significant improvement in expediting problem solving. This difference is quite small, however, and is encouraging but not definitive.
The results generally indicate that time reduction occurred for all participants in the treatment groups, and may have been more significant for persons with less experience in the industry. However, there is not enough of a change in the score to achieve statistical significance.
54

Table 33: Improving Job Effectiveness (Mean Score Comparison)

I am more effective at my job than I was one year ago. [1=do not agree, 10=strongly agree]

COP

N

Treatment 117

Non-

Treatment

98

Mean 6.44
6.04

Std. Deviation
2.28
2.647

Std. Error Mean 0.211
0.267

Table 34: Improving Job Effectiveness (Test for Significance)

I am more effective at my job than I was one year ago. [1=do not agree, 10=strongly agree]

Treatm ent Non-
Treatm ent

Equal variances assumed
Equal variances not
assumed

Levene's Test for Equality of
Variances

t-test for Equality of Means

F

Sig.

t

Sig.

(2-

Mean

df

tailed) Difference

3.45 0.065* 1.201

213 0.231

0.404

1.185 192.742 0.237

0.404

Std. Error Difference
0.336 0.34

95% Confidence Interval of the
Difference Lower Upper
-0.259 1.066
-0.268 1.075
[* p< .1]

55

Table 35: Reduced Time in Completing Tasks (Mean Score Comparison)

Over the past year, I have seen a reduction in the time that it takes me to identify the knowledge I need for my day-to-day work (such as a faster ability to identify the "right person" or "right manual" to consult). [1=do not agree, 10=strongly

agree]

Std.

Std.

Error

COP

N

Mean Deviation Mean

Treatment 124

5.81

Non-

Treatment 101

5.24

2.484 2.829

0.223 0.281

Table 36: Reduced Time in Completing Tasks (Test for Significance)

Over the past year, I have seen a reduction in the time that it takes me to identify the knowledge I need for my day-to-day work (such as a faster ability to identify the "right person" or "right manual" to consult). [1=do not agree, 10=strongly agree]

Levene's Test for Equality of
Variances

F

Sig.

Treatm ent Non-
Treatm ent

Equal variances assumed
Equal variances not
assumed

1.518

0.219

t-test for Equality of Means
t
0.818
0.809

Sig. (2- Mean

df

tailed) Difference

214 0.414

0.279

195.87 0.419

0.279

Std. Error Difference
0.341 0.345

95% Confidence Interval of the
Difference Lower Upper
-0.394 0.952
-0.401 0.959

Table 37: Reduced Time in Completing Tasks (Mean Score Comparison) Junior Tenure, 10 years or less transportation experience

Over the past year, I have seen a reduction in the time that it takes me to identify the knowledge I need for my day-to-day work (such as a faster ability to identify the "right person" or "right manual" to consult). [1=do not agree, 10=strongly agree]

Std.

Std.

Error

COP

N

Mean Deviation Mean

Treatment

(Junior)

47

6.17

2.278

0.332

Non-

Treatment

(Junior)

50

5.8

2.864

0.405

56

Table 38: Reduced Time in Completing Tasks (Test for Significance) Junior Tenure, 10 years or less transportation experience

Over the past year, I have seen a reduction in the time that it takes me to identify the knowledge I need for my day-to-day work (such as a faster ability to identify the "right person" or "right manual" to consult). [1=do not agree, 10=strongly agree]

Treatm ent
(Junior) NonTreatm ent
(Junior)

Equal variances assumed
Equal variances not
assumed

Levene's Test for Equality of
Variances

F

Sig.

3.109 0.081*

t-test for Equality of Means
t
0.702
0.707

df 95 92.507

Sig. (2tailed)
0.485
0.482

Mean Difference
0.37
0.37

Std. Error Difference
0.528
0.524

95% Confidence Interval of the
Difference Lower Upper
-0.677 1.418
-0.67 1.411
[* p< .1]

2.3.5 Performance Outcomes Unique to Centralized COPs An additional finding of interest are the number of performance measures that
demonstrated changes among the more decentralized COP that received treatment (RAID), in comparison to the centralized COP that received treatment (PDT). Specifically, we identify improvements in measures of "community" such as identifying oneself with the COP and identifying the leaders and the goals of the COP. When beginning this research project, it became clear that while members of the COPs regularly share knowledge with one another and/or receive and respond to common listservs, they may not even be aware that they are part of a COP. In order to foster greater knowledge-sharing and understanding of "who to go to" for questions, creating a community "identity" is fundamental to knowledge sharing. Creating a group identity is particularly important for COPs that are spread across an organization, and which may not have a pre-existing "identity" that is created by hierarchy and/or the same location where people work closely with one another.
57

In sum, a number of knowledge-related improvements were noted across the RAID community, but not the PDT community suggesting that the interventions provided more support to loose, decentralized COPs (like RAID) than the more centralized and hierarchical PDT COP. Specifically, the analysis indicates that RAID demonstrated improvements in creating a group identify through the identification of oneself as a member of the COP (Tables 39 & 40), identifying the leaders of the COP (Tables 41 & 42), and understanding the goals of the COP (Tables 43 & 44).
The analysis also indicates that the RAID COP demonstrates improved abilities in solving complex problems, but we do note this change among the ENV and PDT groups (Tables 45 & 46).
58

Table 39: Identifying Oneself with the Community of Practice (Mean Score Comparison)

To what extent do you agree with the following statements?

[1=do not agree, 10=strongly agree] I am a member of the

community of practice on roundabouts/practical

design/GIS/environmental procedures

Std.

Std.

Error

COP

N

Mean Deviation Mean

ENV.S3

29

7.34

2.454

0.456

PDT.S3

56

6.98

2.767

0.37

GIS.S3

75

4.05

3.031

0.35

RAID.S3

68

5.66

2.762

0.335

Table 40: Identifying Oneself with the Community of Practice (Test for Significance)

To what extent do you agree with the following statements? [1=do not agree, 10=strongly agree] I am a member of the community of practice on roundabouts/practical design/GIS/environmental procedures

ENV.S3 PDT.S3 GIS.S3 RAID.S3

Equal variances assumed
Equal variances not
assumed
Equal variances assumed
Equal variances not
assumed

Levene's Test for Equality of
Variances

F

Sig.

1.325 0.253

2.307 0.131

t-test for Equality of Means
t
0.595 0.618 -3.305 -3.32

df
83 63.089
141 140.995

Sig. (2tailed )
0.554
0.539
0.001
0.001

Mean Difference
0.363 0.363 -1.608 -1.608

Std. Error Difference
0.482 0.61 0.487 0.484

95% Confidence Interval of the
Difference Lower Upper
-1.022 0.912
-0.85 1.575
-2.571 -0.646
-2.566 -0.651

59

Table 41: Identifying Leaders of the Community of Practice (Mean Score Comparison)

To what extent do you agree with the following statements?

[1=do not agree, 10=strongly agree] I know who the

leader(s) of my community of practice is (are)

Std.

Std.

Error

COP

N

Mean Deviation Mean

ENV.S3

29

7

2.803

0.521

PDT.S3

55

7

2.789

0.376

GIS.S3

74

4.77

3.329

0.387

RAID.S3

65

6.54

2.716

0.337

Table 42: Identifying Leaders of the Community of Practice (Test for Significance)

To what extent do you agree with the following statements? [1=do not agree, 10=strongly agree] I know who the leader(s) of my community of practice is (are)

ENV.S3 PDT.S3 GIS.S3 RAID.S3

Equal variances assumed
Equal variances not
assumed
Equal variances assumed
Equal variances not
assumed

Levene's Test for Equality of
Variances

F

Sig.

0.119 0.731

7.68 0.006*

t-test for Equality of Means
t
0 0 -3.401 -3.446

df
82 56.829
137 136.29

Sig. (2tailed)
1 1 0.001 0.001

Mean Difference
0 0 -1.768 -1.768

Std. Error Difference
0.587 0.641 0.52 0.513

95% Confidence Interval of the
Difference Lower Upper
-0.81 1.535
-1.275 1.275
-2.796 -0.74
-2.783 -0.754
[* p< .05]

60

Table 43: Understanding the Goals of the Community of Practice (Mean Score Comparison)

To what extent do you agree with the following statements? [1=do not agree, 10=strongly agree] I know what the goals of my community of practice are

Std.

Std.

Error

COP

N

Mean Deviation Mean

ENV.S3

29

7.48

2.278

0.423

PDT.S3

56

6.73

2.963

0.396

GIS.S3

73

4.22

3.115

0.365

RAID.S3

67

5.75

2.803

0.342

Table 44: Understanding the Goals of the Community of Practice (Test for Significance)

To what extent do you agree with the following statements? [1=do not agree, 10=strongly agree] I know what the goals of my community of practice are

t-test

for

Levene's Test for Equality

Equality of

of

Variances

Means

95% Confidence

Sig. (2- Mean

Std. Error Interval of the

F

Sig.

t

df tailed) Difference Difference

Difference

ENV.S3 PDT.S3 GIS.S3 RAID.S3

Equal variances assumed
Equal variances not
assumed
Equal variances assumed
Equal variances not
assumed

4.854 3.702

0.03* 0.056**

1.193 1.296 -3.039 -3.053

83 70.881
138 137.95

0.236 0.199 0.003 0.003

0.751 0.751 -1.527 -1.527

Lower Upper

0.608

-1.065 1.366

0.629

-0.501 2.002

0.502

-2.521 -0.534

0.5

-2.516 -0.538

[* p< .05]

[** p< .1]

61

Table 45: Solving more Complex Problems (Mean Score Comparison)

I can solve more complex problems than I could one year ago. [1=do not agree, 10=strongly agree]

Std.

Std.

Error

COP

N

Mean Deviation Mean

ENV.S3

29

6.59

2.885

0.536

PDT.S3

54

6.17

2.409

0.328

GIS.S3

68

5.09

2.399

0.291

RAID.S3

65

5.52

2.333

0.289

Table 46: Solving more Complex Problems (Test for Significance)

I can solve more complex problems than I could one year ago. [1=do not agree, 10=strongly agree]

ENV.S3 PDT.S3 GIS.S3 RAID.S3

Equal variances assumed
Equal variances not
assumed
Equal variances assumed
Equal variances not
assumed

Levene's Test for Equality of
Variances

F

Sig.

3.134 0.08*

0.078 0.78

t-test for Equality of Means
t
0.705 0.668 -1.059 -1.06

df
81 49.243
131 130.96

Sig. (2tailed)
0.483 0.507 0.291 0.291

Mean Difference
0.42 0.42 -0.435 -0.435

Std. Error Difference
0.595 0.628 0.411 0.41

95% Confidence Interval of the
Difference Lower Upper
-0.764 1.603
-0.842 1.681
-1.247 0.377
-1.246 0.377
[* p< .1]

2.3.6 Performance Outcomes Contrary to Expectations The evidence reveals two areas that were not impacted by the interventions by the
research team in the study. Specifically, there are two elements of the COPs that appear to have improved for the control groups beyond the performance of their "treated" counterparts. We find that the COPs that received support from the research team did not meet any more

62

frequently than their counterparts. Considering the comparison between the ENV and PDT COPs, for example, the ENV was found to meet somewhat more frequently than PDT (Tables 47 & 48). This finding is worth noting since the workshop curriculum, the design of the SharePoint sites and overall interactions with the PDT and RAID COPs were aimed at encouraging more frequent in-person activities among the COPs. Possible explanations for this could be the established practice of regular in-person meetings within the ENV COP prior to the study, or the reliance on virtual communications among the other COPs. Further research could help to explain why the ENV COP is particularly active with respect to the frequency of meetings compared to their counterparts.
The evidence also reveals that the ENV COP reports the most characteristics of COPs among the four groups. A question at the end of the final survey asked respondents to indicate the number of statements that they agree with, with respect to their respective COPs. There were five statements and each one represented one of the five characteristics of a COP. The GIS and RAID COPs scored nearly the same, but the ENV COP scored much higher than their counterparts in PDT. It is possible that the ENV COP had a stronger foundation for COP-type activity before the onset of the study. Yet, the findings encourage further research on the environmental and intervention-related factors that can influence the characteristics of COPs. These findings are outlined in Tables 49 & 50.
63

Table 47: Frequency of In-Person Meetings by COP (Mean Score Comparison)

How often does your community of practice convene (through a workshop, knowledge exchange, training, meeting or some other format) to share knowledge with one another and to build community? (1=at least once per week, 2=at least once per month, 3=at least a few times per year, 4=at least once per year, 5=less than once per year)

Std.

Std.

Error

COP

N

Mean Deviation Mean

ENV.S3

29

2.55

0.87

0.161

PDT.S3

52

3.08

1.326

0.184

GIS.S3

65

3.69

1.172

0.145

RAID.S3

58

3.66

1.052

0.138

Table 48: Frequency of In-Person Meetings by COP (Test for Significance)

How often does your community of practice convene (through a workshop, knowledge exchange, training, meeting or some other format) to share knowledge with one another and to build community? (1=at least once per week, 2=at least once per month, 3=at least a few times per year, 4=at least once per year, 5=less than once per year)

Levene's Test for Equality of
Variances

t-test for Equality of Means

F

Sig.

t

95% Confidence

Sig. (2- Mean

Std. Error Interval of the

df

tailed) Difference Difference

Difference

Lower Upper

ENV.S3

Equal variances assumed

3.134 0.08*

0.705

81 0.483 0.42

0.595 -0.764 1.603

PDT.S3

Equal variances
not assumed

0.668 49.243 0.507 0.42

0.628 -0.842 1.681

GIS.S3

Equal variances assumed 0.078 0.78

-1.059

131 0.291 -0.435

0.411 -1.247 0.377

RAID.S3

Equal variances
not assumed

-1.06 130.96 0.291 -0.435

0.41 -1.246 0.377 [* p< .1]

64

Table 49: COP Characteristics by COP (Mean Score Comparison)
Please check the following boxes based on your perceptions of the work of your community of practice (check all that are consistent with your understanding of your community of practice): [0=0 attributes of COP; 5=5 attributes of COP]

COP ENV.S3 PDT.S3 GIS.S3 RAID.S3

Std.

Std. Error

N

Mean

Deviation

Mean

34

3.4118

1.86056 0.31908

60

2.65

1.97291

0.2547

97

1.3505

1.71416 0.17405

92

1.2935

1.41062 0.14707

Table 50: COP Characteristics by COP (Test for Significance)

Please check the following boxes based on your perceptions of the work of your community of practice (check all that are consistent with your understanding of your community of practice): [0=0 attributes of COP; 5=5 attributes of COP]

Levene's Test for Equality of
Variances

t-test for Equality of Means

F

Sig.

t

95% Confidence

Sig. (2- Mean

Std. Error Interval of the

df

tailed) Difference Difference

Difference

ENV.S3 PDT.S3 GIS.S3 RAID.S3

Equal variances assumed
Equal variances
not assumed
Equal variances assumed
Equal variances
not assumed

1.3 0.257 5.546 0.02*

1.836

92

0.07

0.76176

1.866

72.083 0.066 0.76176

0.249

187

0.804 0.05704

0.25

183.397 0.803 0.05704

0.636 0.41501 0.22903 0.22786

Lower Upper -1.608 0.954
-0.062 1.586 -0.395 0.509
-0.393 0.507
[* p< .05]

65

2.3.7 Preference for Interpersonal Knowledge Sharing to Support the COP The priority given to interpersonal, in-person events is apparent in self-reporting by the
respondents of the surveys. The survey results from survey 3, for example, indicated a preference for in-person trainings and informal meetings, followed by SharePoint-related support (Table 51).

Table 51: Perceived Values of COP Interventions (across all four COPs)

Survey Items: Please score your

preference among the following

activities for supporting knowledge-

sharing in your COP

N Minimum Maximum Rank

In-person trainings

218

1

10

1

Informal meetings

193

1

10

2

Virtual meetings

202

1

10

3

SharePoint for locating documents 206

1

SharePoint for locating experts

(people)

194

1

10

4

10

5

SharePoint for other purposes

195

1

10

6

Mean 7.3 6.18 6.15 5.56
4.6 4.59

Std. Deviation
2.282 2.689 2.369 2.885
2.773 2.545

2.3.8 Survey Data Analysis Summary The evidence of Task 2 explains a great deal about the very real costs of sharing knowledge
within GDOT, and identifies a number of areas that can be improved by supporting COPs. The findings generally suggest that supporting COPs is a worthwhile investment for GDOT, since that support can improve a number of knowledge-sharing and performance-related outcomes. Potential benefits of supporting COPs within GDOT can include the following:
improved levels of expertise in the content areas of the COP; and improved abilities to locate knowledge sources related to their area of work, including:
documents related to their work, and other people and members of their COP.

66

A number of performance improvements were also identified across COPs that were more decentralized, suggesting that the interventions were particularly important for professionals working in more dispersed COPs, which were not tied to the hierarchy of the organization. Specifically, the following improvements were noted among the RAID COP during the course of the study:
identifying oneself as a member of the COP; identifying the other leaders of the COP; understanding the goals of the COP; and solving more complex problems. Additionally, a number of changes were noted across all four COPs, indicating that environmental factors within GDOT have been shaping and affecting the following aspects of COPs in the agency: all members were asking questions of their colleagues more frequently; and members of decentralized COPs asked more questions of one another over the course
of the study. Finally, two issues were found to be stronger among the COPs that did not receive support from the research team, suggesting that other factors in the agency may be supporting COP development, other than the interventions. The survey data analysis indicates, in particular, that the ENV COP holds some of the most frequent in-person knowledge-sharing activities and that they demonstrate the highest level of characteristics of a COP, overall. The survey data analysis generally indicates the treatment had effects on the selected groups when compared to the control groups but these effects are significantly moderated by the initial nature of the group. Effects are different for decentralized COPs than those less centralized groups, when comparing between treatment and non-treatment groups. The differences in effects were noticeable in the changes in the number of questions the members posed to the group and the number of people they contacted. There were also differences in
67

locating written knowledge and locating people. For more centralized groups, both the frequency of questions and the number of people increased with the intervention. However, for less centralized groups, the number of questions decreased while the number of people contacted increased. This suggests that the nature of knowledge exchanges is different and that what is peculiar about these in each group must be explored in order to tailor an intervention to their specific needs.
The awareness of the attributes of the COP was also increased in more centralized groups but not in the less centralized groups. Therefore, it seems a higher threshold for effects of an intervention is present with less developed groups. A one-size-fits-all approach to supporting COP development will not have evenly distributed effects across groups in the organizations.
There were no other statistically significant differences observed with the application of the treatment in the survey data analysis. The differences measured were in the direction of expected improvement from the treatment but were not big enough to be statistically significant. A stronger treatment might include interventions in more areas of activity or with longer duration. This is not surprising given the limitations of this experiment but the results are still encouraging. The lack of statistical significance in some of the comparisons may be due to the small size of the samples of investigated.
2.4 Researcher Observations from Workshops and Personal Meetings
Throughout the course of conducting the workshops for the two treatment COPs, and meeting individually with leadership of those COPs, a number of qualitative observations were also made on the potential benefits of the interventions for the RAID and PDT communities. The importance of hierarchy was evident in advancing activities for the COPs. With respect to generating survey responses, and recruiting attendees for workshop sessions, it was apparent
68

that hierarchical support for the study could generate greater participation in the PDT community as opposed to RAID. This is not a judgment on RAID, but rather an indication that PDT, as a hierarchically oriented COP, has a more direct chain of command that can rally COP participation in emerging activities. Even in the workshop sessions, there was a sense that attendance to sessions was followed out of supervisory expectations in the PDT sessions, as opposed to the RAID sessions. The influence of hierarchy thus offered a strong role in generating participation.
For a decentralized COP like RAID, generating participation involved more long-range planning, and the communication of potential benefits of the sessions that could generate voluntary contributions. Much of the incentive for members of the RAID community to contribute to the study and attend the workshop sessions seemed due to personal interests in seeing a growth in the roundabout knowledge base in GDOT, and an interest to meet with other members of their COP. Attendees of the workshop sessions often remarked that attending the workshops offered a valuable opportunity to "see" who some of the other members of their COP are.
The need for creating a sense of community was apparent across both COPs. In both cases, there was a sincere interest in identifying who was part of the community and who they could reach out to for questions. For PDT, this involved the compilation of instructor and attendee lists from the training sessions. For RAID, this involved simply identifying the broad spectrum of professionals working on roundabouts from across the agency. An interesting observation is that many of the members of the PDT community see each other on a regular basis, as they generally work on the same floor (or nearby floors) within the GDOT's headquarters. The RAID community, on the other hand, is spread out across district offices, illustrated in part by one of the workshops that involved contributions from COP members from
69

across seven district offices. By simply convening the workshop sessions for RAID, in this way, helped to create a sense of community identity that would be difficult to cultivate without inperson meetings that convene members together.
In both cases, it was clear that the leadership of the COPs were interested in further developing communication and knowledge sharing within their respective COPs. The PDT community is working on more explicit updates to their SharePoint sites to improve knowledgesharing, and developed some ideas over the year on how to cultivate more interpersonal communications among members of the COP. The RAID community recently recruited a staff person in the Office of Traffic Operations who will work with the COP's leadership to centralize documentation on roundabouts and for the organization of events for the COP.
2.5 Recommendations
The analysis performed in Task 2 suggests that supporting COPs in GDOT has the potential to improve operational performance for the agency. The very real costs of identifying and utilizing knowledge in knowledge-intensive areas of the agency reveal the impact of knowledge searching on productivity. The benefits to communities, receiving support from the research team, also provide concrete examples of why COPs should be supported in the agency.
The survey data analysis with respect to other qualitative data collected for the study indicates that the types of in-person events that were organized can benefit the development of COPs, and that the benefits will be greatest for decentralized COPs like GIS and RAID. The following table summarizes these conclusions with respect to the different data sources analyzed in Task 2.
70

Table 52: Summary Findings from the Interventions

Survey Data
Notes from Workshop Sessions
Workshop Evaluations
SharePoint Monitoring
Meetings with COP Leadership

Improvements were identified in locating information (identifying documents and persons) and in creating COP group identity (for decentralized COPs), with some encouraging potentials for improvements in employee performance. Simply convening the COPs is important - especially for a decentralized group like RAID. Presenting techniques in knowledge sharing through workshops, as those implemented in this study, improves attention to the COP and the potential adoption of knowledge-sharing practices.
The SharePoint functionality with respect to knowledge sharing was generally new to most participants, and there is a broad interest in online discussion boards. Simply convening the groups, again, offers value to the COPs particularly for a decentralized group like RAID. Holding the videoconferences with district offices for a decentralized COP is particularly valuable. The sessions are useful for modeling mentoring relationships. The sessions generally provided new ideas for knowledge sharing activities that could be incorporated into their COPs' activities. The very nature of convening the sessions helped participants to understand that they were part of a COP, and that there were reasons for sharing knowledge with one another. There was some increase in SharePoint use in the COPs, but much of the spikes were organized around in-person events that were held as part of the intervention. There were little emergent online discussions for the COP, and not an extreme, consistent growth in SharePoint site activity. Simply meeting with the leaders of the COPs from time to time assisted them (especially, Traffic Operations and Roadway Design) in thinking more strategically about the knowledgesharing needs of their COPs, and how they could further support them. Ongoing meetings also helped them to think more strategically about how they could incorporate SharePoint into their COP's needs.

We recommend the following strategies for supporting COPs in GDOT, due in large part to the structures and characteristics of the COPs, as well as, the nature of knowledge that
71

respondents depend upon. Specifically, the importance of tacit, unwritten knowledge across the COPs, indicates the importance of in-person events for fostering and sharing knowledge across the COPs. Examples of activities could include informal activities among members of COPs, such as regular luncheons or coffee breaks. Examples could also include formal trainings on technical areas or SharePoint functionality, with structured programs in place. The expected benefit of ongoing in-person sessions of this kind is the improvement in the understanding of the COP's group identity, and the improvement in locating information (documents and persons) of relevance to the COP.
Since interpersonal communication is so important, there is also value in developing SME lists for each COP, and member lists by expertise, so that professionals can identify relevant experts more efficiently. The SME lists by training area on the PDT SharePoint site are one such example of this approach. In this way, the technology infrastructure can support this aim of building social capital and an understanding of who knows what within the COP.
SharePoint is also an important technology for storing and retrieving documents, coordinating group activity, and for listing members and their respective areas of expertise. The software can provide important support to the COP, but again the importance is on the functions that a virtual platform plays in document storage and retrieval, member identification and communication, and other platforms may achieve similar aims. Online discussion boards have potential, but they require a cultural understanding that 1) asking questions are okay (admitting there are things you don't know), and 2) acceptance that some answers to questions may need some correction. Creating online discussion boards where contributors (of both questions and answers) are anonymous is one such way to experiment with these forums. The online discussions should also be restricted in publicity to include only relevant members, and there should be some "moderators," such as subject matter experts on hand to authorize the
72

responses on the forums. For the PDT COP, the moderators could be the SMEs who lead the trainings; for RAID, the moderators could be peer reviewers internal to GDOT.
SharePoint also has the potential for centralizing document storage and for coordinating events for the COP. In both cases, it is critical to have a content expert on hand to support these efforts, by collecting the needed information and by updating the site to match current needs. The GIS COP has dedicated staff on hand carrying out these roles, and the RAID COP recently hired a person to carry out these responsibilities for their COP.
An additional need for the support of COPs is a high-level champion within the agency who authorizes and supports COP involvement. Andy Casey, for example, was central to encouraging contributions and interests in the study among members of PDT. Gail D'Avino, similarly, encouraged support from Environmental Services. Teague Buchanan and Scott Zehngraff offered critical and valuable support from their respective communities, but the very nature of their COPs may benefit from additional support from a source like the Chief Engineer. Members of COPs are more likely to contribute to SharePoint sites, in-person meetings, and knowledge sharing in general when they see the activities tied to their job descriptions and the overall priorities of the agency. This second issue is even more important for participants in decentralized COPs, like RAID or PDT, that are lacking in internal hierarchy.
73

Task 3: Communities of Practice as a Strategy for Mitigating Knowledge Losses from Retirements
To understand the ways in which retirement is influencing knowledge retention needs, we: 1) examined transaction costs associated with knowledge exchanges to determine if the COPs generate improvements in the accessing of information by GDOT personnel and their strategic partners, and 2) we explored the effectiveness of COPs as a means for coping with retirements or turnover from key personnel within GDOT.
We conducted semi-structured interviews with key personnel approaching retirement. Two to three senior managers who are within 1-10 years of retirement were selected within each group based on the recommendations of other members of the COP. These individuals were chosen because they are nearing retirement, and they were identified by other individuals as active members of the COP, key sources of subject expertise and essential nodes in knowledge communication hierarchies. We asked the individuals questions about their level of expertise, the size and reach of their social networks, the type, frequency and location of knowledge exchanges, and the knowledge retention needs and capacity of their COP (For the interview protocol, see Appendix D). We analyzed the interview data using primary coding of quotes to core concepts to assess the knowledge retention needs associated with retirement, current knowledge practices in place within each community, as well as how the COPs could be used to better support retirement.
Our analysis particularly targeted information about the level and nature of retiree expertise, the size and reach of their social networks, the location of knowledge, which knowledge must be exchanged, and the existing knowledge retention capacities and needs of each of the groups. Our findings are summarized in with these section headings below.
74

We find that much of the knowledge that is of value to the organization and that is being transferred for the completion of tasks and projects is tacit: individual-based and built from experience. Most of the communications within and between groups are about finding the person who has the tacit or experiential knowledge to receive advice about a process or regulation. Each of the groups has initiated programs to retain expert knowledge. The COPs could be helpful in building these initiatives in several ways including by building databases, facilitating communications and enhancing networks within and across groups and divisions. Respondents recommended that GDOT further facilitate the development of COP databases and consider uniting these under a common Department-wide database with a single template. This would require that each individual within the organization complete and update their personal template form yearly. Additionally, many groups identified the importance of seminars and trainings for the development of cross-disciplinary and cross-divisional networks. The Department can support the COP by facilitating cross-divisional seminars and trainings. Finally, managers suggest that the retention of tacit knowledge requires site specific and project-based experiential learning. To the extent possible, the Department should facilitate cross training of senior personnel (so that they can support each other) and training of mid-level managers through shadowing and direct experience with senior personnel. This would help the Department to retain critical, yet fluid tacit knowledge resources.
3.1 Analysis of interview data
3.1.1 Knowledge Source (Repositories: People, Technology and/or Processes) The greatest source of knowledge within the department is people. Senior managers
are the most readily accessible source of knowledge for newer employees and so many of the knowledge requests are directed at people.
75

Everybody reports to me, so pretty much they go to me for everything. I don't think I really have a specialty. And that's one of the odd things about traffic operations it's kind of a big umbrella, you know, of things that go on in the transportation industry; so we cover a whole lot of things, you know, we have access, we have traffic signals, [and] we have intersections and roadways, [and] signs and markings. It is a pretty big umbrella. Manager Traffic Operations
Part of the tacit skill-set managers have is their ability to relate the work that they do to the natural environment. A big challenge of the engineering process is having a solid understanding of the technical and legal requirements of work, and yet being able to adapt these requirements with creativity to the environment. When individuals speak of creativity, they are often speaking of tacit skill sets that are built over time and through experience.
You've got to really think outside the box to be able to get it, to be able to see it. There was a guy that retired last year who worked with me ever since 1987. He was in a design squad with me. One day, we were looking at some aerials and I said, "[Do] you see that stream right there? We have got to make sure that we stay off of it or put an overpass in that low spot right there." He looked at the same picture for five minutes. He said, "There is no low spot and there is no stream." I said, "The stream runs right down there and there's your low spot." He said, "How do you see that?" I said, "It's standing right there." He said, "You see this stuff in 3-D, don't you?" And a lot of times, I did and I can look at an aerial and I can see how the road climbs and loops, so I guess that's just one skill that I have that I can do. RAID Manager
These data suggest that the greatest knowledge resources GDOT should try to retain are
76

tacit and belong to various experts who may be approaching retirement. Below we explore several strategies GDOT might employ to try to retain this type of knowledge.
3.1.2. Knowledge Type (Tacit/Explicit; Complex/Simple) The knowledge demands of engineers are unique because there are both regulatory
requirements as well as technical requirements: It's both technical and regulatory. Like the ADA [Americans with Disabilities Act] compliance, that's regulated. You know, it tells you this is what you have got to have. It has got to be 27 inches of clearance underneath the desk. It tells you that something can't stick out from the wall more than 8 inches, because you still have to be able to reach the end. It tells you the grade of the wheelchair ramps. So, that's when you start to get in the technical side. But it also tells you the regulatory side, it tells you this is how you will build it. Member of the RAID Group The ability to design projects through technical and legal specification and yet be able to
adapt the projects to a contingent and living environment requires considerable experience and expertise. The strength of senior managers is their expertise and tacit knowledge. This was acknowledged in the data in a variety of ways.
Specifically, I am a catch-all person here. I mean, I have got that institutional knowledge of cultural resources, archaeology... so, I am fielding questions about that...I am a direct liaison to consultants and project managers as well at that point. Then probably, the primary job is to ensure that when we have projects that we call the let, the letting which is letting to construction, we make sure those projects get certified for the letting, which basically means we got all the environmental information completed, we got necessary permits that we need, and it is ready to be advertised to be built. Environmental Manager
77

Managers have to manage both people and work flow. The experts in greatest need of retention regularly are situated at the interface of codified knowledge (standards, requirements, process knowledge etc.) that is located in documents and personal knowledge of subject matter experts as well as of adaptation and creativity in the past. So these experts link people with people in the organization as well as people with important technical and regulatory documents, standards and processes.
Well, at GDOT, we have got the position of a director reserved for a person who manages multiple offices. I manage the office and then have multiple Project Managers, and two different groups under me and multiple teams underneath those; so it's kind of like when they say we're office administrators, that's really what we're doing. We have a specialized function that our office does. GIS Manager
3.1.3 Transfer Process (Hierarchy or no hierarchy) The organization still has considerable hierarchy and many engineers feel it is important
to work up the chain of command when working through challenges. However, increasingly, the hierarchy is not only vertical but also horizontal. The more senior employees tend to be better connected outside of their area of expertise. This is important not only for working across the organization, but also for working outside of the organization with other agencies and with consultants.
I mainly manage consultants, so I have to tell them how to do their job, but as far as internally, I have to keep my supervisor informed of what's going on, and if any of our collateral duties happen to touch estimating or field plan reviews, I'll just go to the consultants personally and tell them what's going on. PDT Manager
78

Contract work and consulting are additionally changing the workflow and knowledge retention needs and capacities of the organization.
Four years ago, I used to have an office that had 84 people. Today, I have an office with 33 people. And I have lost my traffic data collectors. They used to go out and do all the traffic counts, but those have been replaced with contract work. Then within the last year, we watched all our road inventory people retire or walk out. So we're looking into contracts in the context of local governments to try to replace that. So once upon a time, we used to have a very robust local road program, and now today, we're looking at how we replace that with a series of contracts. GIS Manager
Senior managers have to reach out of the organization to manage contracts. This temporally and spatially shifts work and expertise away from GDOT. Rather than experts who are associated and organized within the organization, GDOT must try to retain knowledge that is flexible and flows through the organization. Even if GDOT wants to train and develop experts for specific tasks, the nature of contract work makes this task very difficult because contract decisions are based largely on budget rather than prior history. Contracting also negatively affects work morale as jobs that use to be stable within the organization are replaced by temporary work.
The first impact about contracting that one has is negative. Employees feel the contractors are coming to take away their jobs. And I can tell you that feeling is universal - no matter what particular area they are from. People who have done, say, striping for 15 years or for the past 10 years are upset because we are bringing in people who are doing their jobs, so they perceive their jobs as gone. GIS Manager
79

These negative associations with contract work affect employees' ability and desire to connect and share knowledge.
We have to prepare the employees. Let's say, contractors are coming in because we don't have enough staff, but what we want our employees to do is not only share their information but to learn from the contractors. We have a lot of GIS consultants and they have just come out of college. Well, they actually know more GIS than some of our GIS people. So I want our people to sit there and to work on this project with a contracting expert so that both can learn something new. It's not like the contractor is coming in to replace but coming in to help our employees learn and to learn from our employees. GIS Manager
Contracting may be beneficial if the contractors have specialist knowledge they can share, but the challenge is to build a process that incorporates expert knowledge through contracting rather than a process that is designed to perform work. We explore ways in which this might be accomplished below, but one strategy might be for GDOT to increase the number of educational and topic specific seminars that incorporate both core employees as well as contractors.
3.1.4. Frequency and Types of Communication (Processes of validation, if any) Consulting creates the need for process verification with senior or mid-level managers
who hold tacit knowledge within GDOT. But now we have two consulting firms that are national experts on roundabouts and one of them, actually both of them do it, but one mainly, will call me and let me know they'll be in town and ask me to stop by so I can cover certain aspects or maybe answer questions. They get into the design part of a roundabout, but they don't get into the 80

constructability or how to build it. That's my specialty. How to keep traffic flowing while you build a massive circle out there is my specialty. RAID Manager
This is particularly the case for design engineers who must mitigate technical and legal specification while making projects work in a contingent environment. This was particularly a challenge addressed in the context of successfully meeting environmental standards.
We get a fairly interesting and wide range of questions from people. One of the big questions is just how we get from A to B when we are working on a concept. What are the rules that govern doing this part or that part of the process? I think, in general, it's probably trying to figure out how to get through the environmental process. How do you negotiate it while you're balancing that and the engineering of the project? PDT Manager
The challenge of meeting technical and legal environmental requirements is very difficult in the context of contract engineering. It is these issues that internally and externally demand knowledge located within experienced and technically skilled engineers. There have been questions within the organization about the ability of databases to manage the type of tacit knowledge that is most valuable to the organization.
It's hard to put that kind of knowledge into a database. Yeah, I guess you could, you could put it into a database and that's what we're beginning to work on even with the NEPA section - it's like frequently asked questions. For example, say you're working for me and you have an issue on 6F, and you don't even know what 6F is. However, you're
81

the NEPA person [and] you don't need to know what 6F is. So internally, you have a database where all you have to do is type 6F and there should be some information that we have encountered and worked through before [so] that we can give you a start. Environmental Manager
There is some sense in which the technology can be useful in building and storing data about who knows what. This is particularly envisioned in the form of wiki-databases that update core concepts and project histories. Technology is also useful in helping experts visualize what they need to work on before or without going into the field.
When you're putting your traffic counts, your vehicle counts into the software, GDOT software, it helps you visualize how traffic is going to flow through and around a roundabout. If you have a multilane, you split up the traffic in a certain way and it helps to see how it's going to work. A lot of software seem to be like that. It [software] gives you kind of indications, it kind of confirms what you may already believe, or leads you in a different direction, or gives you other things you need to look at. RAID Manager
Additionally, many individuals describe the potential of databases built from video-logs of retiring experts. These would have associated keywords attached and allow individuals to not only look at documents but to watch recordings of experts who describe projects they have worked on. However, both of these initiatives require dedicated staff to update and manage the databases.
What's painfully obvious is that you almost need a dedicated staff to do that type of work at that point. I can go to Chad, who's kind of our video expert that does our video work, we've got a camera that's top of the line, but I mean ultimately it's still finding the
82

time in my busy day, in his busy day, the person who's retiring, to sit down and talk with him. And so that became painfully obvious in our taskforce group. I don't have time to do that. I can say that I can do that and I might be able to get an interview for you, but as you know, there are several other steps after that. You have got to then transcribe this, you've got to filter out what's good and what's bad, and that takes time. And that's why you almost need a dedicated staff that that's the job they do at that point. Environmental Manager
3.1.5. Effects and Consequences (Processes or methods of retaining the knowledge) There were several initiatives under development in each of the groups we surveyed.
These ranged from academies that use seminars to train new hires, to informal one-on-one mentoring for junior engineers. For example:
[In] this office, we're doing what they call a Traffic Engineering Academy. So, we're taking pay grades 13, 14, and 15 and are basically doing a rotation through each one of the sections, and I'll have one person who works [and has been] trained and mentored [by me] basically. It is a mentoring program. They still do their other job, and [train] when they get some slack time. Because our office is small, we can't just say that he's yours for the next two weeks. His job would fall [and] have gaps in it. So, we designed it to where the mentoring program was spaced out to where you could do three or four hours here and then do your job for three or four hours. That way we get to where the knowledge is transferred through to the lower level engineers. RAID Manager
Initiatives such as the Traffic Engineer Academy allow senior personnel to directly train and mentor mid-career managers.
83

I mean, GDOT has got, in my opinion, one of the best training facilities. The new area is pretty nice because it has computer labs and everything. The training opportunities are great. As a matter of fact, just for the month of May, you could have 5 different management classes. You could have classes on hydrology; you could have classes on roundabouts. If you don't take advantage of the training provided to you for free, then you're missing out. RAID Manager
However, these types of training seminars could focus more explicitly on site-specific knowledge.
I think it is helpful to go out with people. To me, it is also helpful to have a group go out, look and have conversations about what they think is going on in a specific site. Especially, if you have a bunch of new people to a job, it is useful to go to a site and have someone like me to listen to those comments and to encourage certain site-specific issues. For example, with respect to an intersection project, we could go out there with them and tell them these are the things you're looking for with this intersection. Mentoring and explaining how I look at things, I think that helps a lot, because I have learned over the years how I need to look at things. RAID Manager Site-specific trainings and visits with junior and senior personnel can help to transfer knowledge. Trainings are another effective knowledge retention strategy. These initiatives tend to be focused on mid-level managers and allow them to not only develop new knowledge but to connect to individuals across the organization. We would recommend not only bolstering these sorts of activities but expanding them to include, junior, senior as well as contract employees.
We're starting, or getting ready to start doing more of what we call cross-training where people go out with each other to get a better understanding for everybody of what the
84

other disciplines have to do at that point. But that's tough when you're shorthanded. When you're over worked now, it's hard to dedicate a day or two from your work to learn about ecology and stuff like that. [Interviewer: so this is something new that's been happening, trying to build cross experts...] Right now what we're trying to do is we have internal training once a month. And so once a month, we've got a different specialty that goes in and everybody can be there. That internal training is going to be nothing but air, and so those air specialists come down and tell everybody about his or her job, what they need to know to do their job, [and] what the end result is. And so next month, it will be history, and history will get in there and they'll tell one aspect of their work. And so each month, we rotate with a different specialty, and so at the very least, you're getting internal training. Environmental Manager
Finally, several individuals spoke about the use of manuals as an effective strategy for organizing and attempting to retain expert knowledge.
In our office, this new initiative of knowledge sharing with Robert [is] getting us [to create] a book of how we do our job. I think that's a great first step, but there's not been anything like that before. Once I graduated from college and saw how the Department operated in the `90s, we still had some older guys that would just like [to] hoard knowledge and it felt like that was their ticket to retirement and they weren't really going to share with a bunch of people. They felt like they needed to really compartmentalize this so they would be a resource and I saw that a lot in different areas. PDT Manager Having manuals and databases is seen as a way to de-compartmentalize knowledge and expertise, and serves a means for expediting knowledge search and retrieval.
85

3.2 Recommendations
3.2.1 Level of expertise of retirees Managers and directors within the four groups demonstrated high levels of both
specialist knowledge related to their particular discipline or practice, as well as generalist managerial knowledge required to organize groups and projects. Managers in the RAID group demonstrate high levels of what we term `technical jurisprudence'; they have to balance the technical and legal requirements of state engineering projects and adapt these requirements to the physical and cultural landscape of the particular engineering site. Managers in the PDT Group are responsible for the workflow through the Plan Development Process (PDP). They interact with individuals across the organization and have strong process based knowledge and skills required to move projects through each phase. Managers in the ENV group tend to have a disciplinary focus such as ecology, archeology or cultural resources. In addition, they are responsible for various aspects of the PDP and oversee multiple projects. Interaction with a wide network of agencies is essential to what they do. Managers in the GIS groups have specialist knowledge of information systems. They are also active mentoring younger staff members to build networks across GDOT and outside of the agency. Networking and integrating knowledge between regular staff and consultants is also a critical function of managers in the GIS group, especially as the amount of contract based work increases.
In addition to these particular skills, near retirees in each of the groups have strong managerial and leadership skills built through years of mentoring, training, organizing groups and operating projects. Most of these skills and knowledge are tacit: they are built through years of practice and experience and are difficult to transfer. One of the biggest knowledge retention challenges for GDOT is transferring these tacit skills and knowledge before retirees leave. Nevertheless, managers in each of the communities, particularly the ENV and PDT
86

groups, expressed a strong desire to contribute to the community and to mentor younger staff members. The COPs can help support these efforts by providing informal communication networks through which individuals can locate each other. In addition, formal training including mentorship and shadow opportunities across the groups will be useful to retain tacit skills and knowledge. Finally, programs which take advantage of SharePoint and leverage existing initiatives that develop databases of expert knowledge (in the form of reports of critical job functions, project-based experiences and specialist skills) would help to provide a means to search and identify tacit resources even after individuals have retired.
3.2.2 Size of social networks Mangers in each of the groups deal with and benefit from extensive networks that
include the GDOT community, specialist resources, federal agencies, and the public at large. The nature of these groups varies somewhat based on the nature of the work conducted, as well as the way in which the offices of the group are situated. GIS and RAID are the most dispersed COPs among the four under examination. Their members come from offices and divisions across GDOT, which makes cohesion amongst the groups and the exchange of tacit knowledge difficult. Most of the ENV members are physically located within the headquarters, making it easier for them to communicate directly with one another. Nevertheless, the nature of their work is interdisciplinary and the group interfaces with virtually every office and project phase due to the nature of legal requirements. Communication between the disciplines and across the agency is essential to the success of projects. With its oversight functions, the PDT group performs a similar function. The PDT group is also located in a single office, which offers greater cohesion amongst group members. However, PDT and GIS deal extensively with consultants, such that essential knowledge and skills are routinely outsourced and difficult to retain within GDOT.
87

Within each of the groups, various training programs have been identified. To further build cohesion and advance social networks within and between the communities, further organization-wide events and trainings should be held. These can be focused on particular subjects or specialist knowledge. Eventually, once the SharePoint sites become sufficiently advanced, it would be useful to bridge them with a single database, which would store expertspecific knowledge and provide a source of networking and community development. Several managers envision this as an open source, searchable organizational Wikipedia. A singe template could be developed, with core characteristics such as job function, knowledge competencies, types of projects completed and significant project stories. Each of these sections would have searchable tags. Staff members would be encouraged to complete the template and to use the database to identify knowledge sources. GDOT could encourage individuals to update and add to their template yearly. This would over time fully develop database material, as well as encourage the engagement with various COPs, linked through the database.
3.2.3 Type, frequency and location of knowledge exchanges: technology, people, or documents
We asked retirees to identify the type of knowledge queries they receive and the primary source of knowledge within their operations, whether people, technology or process. Within the RAID group, manuals and reference books are the primary source of knowledge. Much of the mentoring within the group is focused around teaching through manuals, or helping junior staff to identify the correct people who have the knowledge necessary to complete a task or project. However, several managers noted that the application of knowledge is transforming from person-based to technology-based, and that this creates additional challenge of training staff members to understand the rationale behind processes and to retain
88

knowledge rather than relying on technology to complete particular tasks. Members of the ENV group shared a similar perspective that knowledge is person-based and much mentoring is about locating the person with the specialist or experiential knowledge. The members of this groups suggested that the database they had initiated for cataloguing personnel expertise could further be augmented to help with knowledge location. Within the PDT and GIS groups, there was also strong concurrence about the significance of finding person-based knowledge; however, there is also a need for understanding technical process. Within the GIS group, there was recognition that technology can help to augment personal knowledge by facilitating a broader understanding of a project or set of projects.
Finally, across the groups, there was a strong recognition of the importance of tacit and site-specific knowledge, which are gained through experience and engagement on site. Managers identified tacit knowledge, what they at times referred to as `thinking outside the box, creativity, or site-specific application of technical and legal requirement' as the most valuable and difficult form of knowledge to develop and transfer. In addition to facilitating the function of the groups and allowing for the development of complex engineering process, managers suggested that tacit and site-specific knowledge could help to boost morale and give staff members a sense of ownership and pride in their work. Particularly within the RAID and ENV groups, there was also a recognition of the importance of these two forms of knowledge in linking projects with the community and ensuring that they have beneficial elements beyond just providing infrastructure and transport capacity.
These findings support the utility of a database that would link the COP and help individuals identify sources of specialist knowledge or particular experience. It is difficult to transfer or exchange tacit and site-specific knowledge through a database, but it would at least help staff to identify the correct people to speak to about particular topics. In addition, several
89

managers suggested that more training on-site and experiential learning through mentorship and shadowing is needed to build tacit and site-specific knowledge. This could be pursued within and across the COP.
3.2.4 Knowledge Retention Capacity Each of the COP recognized important knowledge retention capacities through
programs that exist or are being initiated within their group. The RAID group has access to the Traffic Engineering Academy through which mid-level managers are rotated between sections to learn various skills and to build networks within the Department. This and routine subject specific trainings were cited as key programs for knowledge exchange and retention. Members of the group however identified the need for site-specific trainings and further shadowing opportunities. Within the ENV group, there is an initiative to build a database that records personnel knowledge through interviews. This program is in need of further staff and support, but could be useful in developing a Department-wide database for knowledge retention. The ENV group likewise uses seminars and trainings to develop interdisciplinary networks and knowledge. The ENV group also tries to build cross-specialty expertise (between ecology and resources for example) by having senior managers rotate job responsibilities. This duplicates organizational knowledge and means that if one of the managers leaves, his or her knowledge can be replicated by another senior manager. Such a program would be helpful to produce similar results in other groups, and across the Division.
Within the PDT group, there is a similar initiative to build a job specification manual, which catalogues the expertise of each group member. The group has developed succession planning which involves mentorship between mid-level and senior managers, but could use additional support to provide opportunities for cross training. Finally, within the GIS group, there are also weekly seminars and trainings on specialist knowledge, which not only
90

disseminate knowledge, but also help individuals to network and identify critical sources of knowledge. The group is augmenting this program by developing a database to catalogue and store personnel knowledge.
These programs help to support the development of the COP, to facilitate the exchange of knowledge and the creation of networks, and to duplicate and retain expert knowledge within the Department. However, members of each of the groups suggested that additional support and staff are needed to make these programs successful.
Since much of the Department's tacit knowledge is based on past experience, knowing what was done and by whom and how challenges were resolved is essential to the function of the Department and to the retention of critical tacit resources. All of the groups have some form of seminar training, site/context specific shadowing and mentorship opportunities, and developing knowledge databases. To the extent possible, these initiatives should be supported, replicated and linked across the COP through Department wide opportunities.
3.2.5 Overall Retirement Recommendations We recommend that the Department focus on three initiatives to facilitate the retention
of expert knowledge from near retirees: 1) the construction of person-based knowledge databases, 2) cross-divisional seminars and trainings, and 3) site-specific and experiential learning opportunities with senior personnel. These activities should be targeted first on personnel identified as nearing retirement, and then to the extent possible, across the organization.
Each of the COP groups are developing databases to catalogue person-based knowledge and expertise. These initiatives take various forms, from manuals to websites. Such initiatives should be supported and potentially consolidated under a single Department-wide SharePoint database. Such a database would have a common template to be completed by each employee
91

and updated on a yearly basis. The template could for example include core characteristics such as job function, knowledge competencies, types of projects completed and significant project stories (either written record or video log). Each of these sections would have searchable tags. The database would help individuals identify critical tacit knowledge sources, as well as through video logs or brief written recordings store the tacit knowledge of individuals who may soon retire from the Department.
In addition, the COPs have demonstrated considerable success with knowledge sharing and network building through various forms of specialist knowledge seminars. These efforts should be supported not only within the groups, but also across the groups. The seminars are often conducted as brown bag lunches, but could also be successful after work if paired with a social gathering. Such cross-division seminars and trainings would augment seminars within the COP and facilitate networking across the organization.
Finally, members of the COP have suggested that site-specific training and shadowing opportunities are critical to the dissemination of tacit knowledge. Such training (to build specialist or managerial knowledge) often takes the form of shadow assignments, where a member of staff shadows another member of staff for a period of time to learn first-hand how they perform their duties and tasks. The shadowing should initially focus on pairing mid-level managers with senior managers to retain the knowledge from senior personnel who are close to retirement. However, shadow assignments could also be supported across divisions to duplicate specialist and managerial expertise and to further develop networks. This model has worked very successfully within the ENV group to duplicate expertise among senior personnel and through the Traffic Engineering Program to build cross-disciplinary expertise. Staff exchanges also help to support network development and the retention of tacit knowledge and should be implemented across rather than just within divisions.
92

Conclusions and Recommendations
The results from this study explain the nature, means for supporting, and functions of COPs within GDOT. Specifically, the findings provide an evidence-based understanding of the actual dynamics of COPs within the agency. By analyzing the behavior and functions of COPs over two years, we can also draw conclusions about how to support knowledge sharing in the agency. The objectives of this study were three-fold:
1. To examine more closely the set of existing patterns of knowledge exchanges within GDOT and determining the conditions for their development into fully operating COPs by means of pilot demonstration sites;
2. To monitor changes in performance, transaction costs, and productivity from the Introduction of COP; and
3. To explore the effectiveness of COPs as a means for coping with retirements or turnover from key personnel within GDOT.
4.1 Patterns of Knowledge Exchanges and Conditions for COP Development The examination of the existing patterns of knowledge exchanges revealed that there
are interactions among groups in GDOT that have several features of COP. However, there are important differences in their characteristics having to do mainly with whether their knowledge exchanges are mostly centralized or decentralized. These differences led also to differing constraints on their potential for greater development. For example, the less centralized COPs (RAID and GIS, in this study) seem to face some of the greatest challenges in knowledge sharing. The main reasons are:
1. Cross-division structure of their memberships; 2. Diminished support of formal authority in their communities due to the absence of
formal hierarchy; 3. Knowledge of the community seems to play a more indirect role in the daily work of
their members; and 4. Lower levels of reported expertise among the members of decentralized COPs.
93

Working across boundaries of divisions and offices within the organization creates higher barriers for informal exchanges. It demands more tools and systems to provide a structure since the decentralized COPs lack many of the controls inherent and readily available for informal exchanges that work along the lines of the organizational hierarchy. These lead to the first two recommendations: Recommendation #1: Identify explicitly specific knowledge-sharing activities to increase the awareness of existence of a community that shares knowledge across the boundaries of divisions and offices. Recommendation #2: Assign a specific staff person (or persons) to each decentralized COP to provide administrative support in collecting documents, updating online platforms, and coordinating group events. Insofar as the decentralized COPs we identified and studied in this project did so, the advantages were obvious. For example, the RAID COP has hired a person to help with the coordination of their group, and the GIS COP has dedicated staff for similar roles.
Since formal authority is not directly aligned with the knowledge exchanges, uncertainties about the appropriateness or priority of sharing knowledge at specific instances may deter some from engaging in potentially fruitful exchanges. Even though these may seem to be isolated instances, these are network phenomena and, therefore, benefit (and suffer from) network effects. That is to say, the value and impact of each individual action and link increases exponentially with the size of the network. Barriers to these activities must be reduced as much as possible to enhance the impact of the existing network. This leads to a third recommendation: Recommendation #3: Provide senior-level support for the knowledge-sharing activities that constitute a COP.
This will communicate to members the tangible reasons for contributing to the COP and demonstrate the link between the COP and strategic priorities.
94

In spite of the fact that COPs with a more centralized structure work along and take advantage of the existing hierarchy of the organization, there is room for improvement in these as well. The remaining dimension that would benefit both centralized and decentralized COPs is increasing the ease with which COP members find each other. Before we enter the realm of structured interventions to develop COPs, the importance of improving informal interactions should be highlighted. This leads to a fourth recommendation: Recommendation #4: Organize and/or facilitate some in-person, interpersonal forums for sharing experience-based knowledge.
These needn't be very sophisticated or costly events. They might be extensions of occasions that already exist in which staff "talk shop," in which small adjustments might make a big difference in how much more collegial interaction takes place. Part of the role of leadership in the organizations might, arguably, be said to improve the lines of communication in the organization to stimulate cooperation and team work. COPs are ideal for achieving the benefits of synergy mostly using resources the organization already has.
4.2 Performance Improvements from COP Support Interventions The findings of this project also indicate that more structured support of COPs is a
worthwhile investment for GDOT, since that support can improve a number of knowledgesharing and performance-related outcomes. Participants who were involved in the study and were subjects of the interventions administered as part of the design identified specific performance-related outcomes. The data analysis revealed that these outcomes occurred to a greater extent for COPs that received the intervention and the difference between those receiving the intervention and those that did not was statistically significant. Therefore, there is good empirical basis for recommending support that follows some of the patterns of the interventions carried out in this project. Among those performance-related outcomes were:
95

Greater ability to locate needed written documents; Greater ability to locate appropriate experts within the COP; Enhanced ability to identify more accurate knowledge for a given problem; Improved job effectiveness; and Reduced amount of time to complete tasks.
In addition, some of improvements in performance due to the intervention were greater for members of the decentralized COPs than for the more centralized or hierarchical communities. This leads to the following recommendation concerning performance improvements through COP: Recommendation #5: Support more decentralized COPs first since they have the potential to generate greater improvements in performance at similar costs.
The areas in which these differences were greatest (all found to be statistically significant improvements compared to members of COPs that did not receive interventions administered by the research team) were the following:
Clearly identifying leaders and understanding the leadership structure of the COP; and Better understanding the goals of the COP. The design of the intervention and the results provide a pattern from which to address a structured approach to support the development and improved impacts of knowledge sharing in COPs. First, decentralized COPs rely more upon voluntary leadership and coordination that has inherent weaknesses and uncertainties. Some dedication of staff time and indication of importance from senior leadership can go a long way in compensating for this disadvantage. Second, the most important type of knowledge shared in the interactions within COPs is tacit knowledge acquired from experience and embodied in the persons that have done similar things before and solved similar problems. This is unwritten knowledge dispersed across various parts of the organization. The only way to access this sort of knowledge is through person-to-person contact in informal settings. These are, for example, "water cooler talk", side conversations in
96

meetings called for other purposes, office group luncheons, among other such occasions. More focused activities, such as trainings on technical areas or SharePoint functionality, with structured programs in place would serve the double purpose of providing skills in the use of support infrastructure and creating opportunities for informal exchanges. The expected benefit of ongoing in-person sessions of this kind is the improvement in the understanding of the COP's group identity, and the improvement in locating information (documents and persons) of relevance to the COP.
Since identifying and locating SMEs was another important benefit of enhancing the COPs with support, it follows that lists of experts can also be created and reduce the time needed to identify individuals that are most suited for the sort of expertise a problem requires.
On support technology, SharePoint, offers a number of tools and resources that enhance opportunities for knowledge-sharing in the COPs, including: publicizing membership lists of the COPs, scheduling upcoming events, facilitating discussion boards, and creating space for documents storage. From these results, a recommendation for a strategy to provide structured support of COPs can be offered. Recommendation #6: The recommended strategy for supporting COPs in GDOT includes the following components:
1. Identify the membership of COPs by consulting with the groups' leadership; 2. Achieve senior-level agency support for the COP to communicate to the group the
importance of knowledge-sharing activities in order to link COP activities to strategic objectives; 3. Dedicate staff time to assist with organizing and coordinating COP activities; 4. Select and conduct in-person knowledge-sharing activities on a regular basis to facilitate the sharing of tacit knowledge; 5. Meet regularly with the COP leadership to evaluate goals, objectives, and needs for the COP; 6. Identify and publish contact information for SMEs; and 7. Utilize SharePoint to store and share information related to the community.
97

4.3 Retaining Knowledge from Agency Experts Nearing Retirement The COPs have an added advantage in that they also provide important support for
sharing knowledge with agency experts nearing retirement. All the tasks of this project highlight the importance of experience-based, tacit knowledge. These experts generally have long time experience with the agency and have developed superior judgment on topics that require understanding how criteria from different jurisdictions interact in specific types of problems. For example, most transportation projects involve the resolution of technical problems in the context of applicable regulations. The latter change over time and have different provisions for a variety of circumstances that may not differ too much on the nature of the technical problems. Only very experienced members of the agency have the perspective necessary to capture the nature of these interactions and know how to align the technical and regulatory guidelines for the projects at hand.
In a similar vein, since most transportation projects now involve the work of consultants and contractors and complex contracts involving not only deliverables but also procedural issues are required to carry out such projects, the ability to integrate knowledge to consultant and contractor management is critical for success. This is the second key area of crucial expertise that senior agency experts nearing retirement have and an area in which the agency becomes vulnerable as it loses them.
The findings of this project on the existence of COPs in GDOT, their features and the most effective ways to enhance their effects, contribute directly to the issue of capturing knowledge from senior agency experts nearing retirement. As the experience of long time agency experts differentiates over time as each individual is part to a specific combination of problem resolution experiences, the knowledge becomes more and more identified with the individual himself or herself. This leads to the following recommendation:
98

Recommendation #7: Create person-based knowledge databases that cover most of what a key agency expert nearing retirement knows for his or her daily responsibilities.
This can be achieved with debriefings and personal histories of project experience that highlight the specific type of superior judgment the individual contributes to the agency. Recommendation #8: Hold cross-divisional seminars and trainings that will highlight the nature of the superior judgment of these experts and facilitate the development of the new generation of staff with similar abilities.
Since most of the critical knowledge of these senior experts is the broad perspective they bring to problem resolution, these sorts of trainings, while not frequent, may be very important in allowing the next generation of senior staff to envision the kind of knowledge their leaders have acquired. These sorts of events may serve as a way to gauge who might be the most promising candidates for mentoring in this direction. Recommendation #9: Provide site-specific and experiential learning opportunities with senior personnel.
This is another activity targeting the transfer of tacit knowledge. Since it is mostly transferred by sharing the actual activity and acquiring experience in it, these are ideally suited for critical problem and site-specific skills.
These activities should target first, as the source of content, personnel identified as nearing retirement, and then to the extent possible, senior personnel a little further down the line across the organization. Databases are critical for identifying employees with extensive and unique knowledge in the agency. Being a member of a specific community also supports the creation of group identity in the COPs. Cross-divisional seminars and site-specific experiential learning provide opportunities for the sharing of tacit knowledge, and help learning employees who are "learning" new information to understand the means of applying the new knowledge particularly as that understanding relates to technical regulations, and contextual constraints.
99

4.4 Concluding Remark The main results from all three tasks improve our understanding of the roles and
purposes of COPs in GDOT. Specifically, the evidence of this two year study identifies how knowledge-exchanges and COPs have "emerged" in different divisions in the agency. We also identify the critical role that COPs and related in-person knowledge sharing practices have in capturing and disseminating tacit knowledge. The results indicate that activities of this kind can be improved in GDOT, and that identifying and supporting COPs can yield performance improvements for the agency. The increased use of COPs can improve knowledge retention in the agency, and improve the overall capture of experience-based knowledge among agency employees nearing retirement.
100

References
Badke-Schaub, P., & Frankenberger, E. (1999). Analysis of Design Projects. Design Studies, 20, 465480.
Brown, J. S., & Duguid, P. (2001). Knowledge and organization: A social-practice perspective. Organization Science, 12(2), 198-213. doi: 10.1287/orsc.12.2.198.10116
Brown, John Seely, & Duguid, Paul. (1991). Organizational Learning and Communities-of-Practice: Toward a Unified View of Working, Learning, and Innovation. Organization Science, 2(1), 40-57. doi: 10.1287/orsc.2.1.40
Cheng, T. T., Wang, H. S., & Wang, L. B. (2009). A Study of Tacit Knowledge Transfer Based on Complex Networks Technology in Hierarchical Organizations. In J. Zhou (Ed.), Complex Sciences, Pt 2 (Vol. 5, pp. 1485-1494). Berlin: Springer-Verlag Berlin.
DeLong, David W. (2004). Lost Knowledge: Confronting the Threat of an Aging Workforce. Oxford: Oxford University Press.
DeLong, David W., & Davenport, Thomas. (2003). Better Practices for Retaining Organizational Knowledge: Lessons from the Leading Edge. Employment Relations, 30(3), 51-63.
Egbu, Charles O., & Robinson, Herbert S. (2005). Construction as a Knowledge-Based Industry. In C. J. Anumba, C. Egbu & P. M. Carrillo (Eds.), Knowledge Management in Construction. Oxford: Blackwell Publishing.
Elliott, Robert H. . (1995). Human Resource Management's Role in the Future Aging of the Workforce. Review of Public Personnel Administration, 15(2), 5-17.
Grewal, DS, & Haugstetter, H. (2007). Capturing and Sharing Knowledge in Supply Chains in the Maritime Transport Sector: Critical Issues. Maritime Policy and Management, 34(2), 169183.
Hildreth, Paul M., & Kimble, Chris. (2004). Knowledge Networks: Innovation Through Communities of Practice. Hershey, PA: Idea Group Publishing.
Lave, Jean, & Wenger, Etienne. (1991). Situated Learning. Cambridge: Cambridge University Press. Nobbs, H. (1993). Future Role of Construction Specialists. Pan, S. L., & Leidner, D.E. . (2003). Bridging communities of practice with information technology
in pursuit of global knowledge sharing. Journal of Strategic Information Systems, 12(1), 71-88. Polanyi, Michael. (1967). The Tacit Dimension. New York: Anchor Books, Doubleday & Company, Inc. Raven, A. (2003). Team or Community of Practice: Aligning Tasks, Structures, and Technologies. In C. B. Gibson & S. G. Cohen (Eds.), Virtual Teams that Work: Creating Conditions ofr Viritual Team Effectiveness. San Francisco: Jossey-Bass. Robinson, H.S., Carrillo, P. M., Anumba, C. J., & Al-Ghassani, A. M. (2001). Linking Knowledge Management to Business Performance in Construction Organizations. Paper presented at the ARCOM 2001 Conference, Salford, United Kingdom. Rubenstein-Montano, B., Liebowitz, J., Buchwalter, J., McCaw, D., Newman, B., Rebeck, K., & Knowledge Management Methodology, T. (2001). A systems thinking framework for knowledge management. Decision Support Systems, 31(1), 5-16. doi: 10.1016/s01679236(00)00116-0 Shadish, William R., Cook, Thomas D., & Campbell, Donald T. (2002). Experimental and QuasiExperimental Designs. Boston: Houghton Mifflin Company.
101

Sheehan, Tony, Poole, Dominique, Lyttle, Ian, & Egbu, Charles O. (2005). Strategies and Business Case for Knowledge Management. In C. J. Anumba, C. Egbu & P. Carrillo (Eds.), Knowledge Management in Construction. Oxford, UK: Blackwell Publishing.
Snyder, William, Wenger, Etienne, & de Sousa Briggs, Xavier. (2004). Communities of Practice in Government: Leveraging Knowledge for Performance. The Public Manager, 32(4), 17-21.
Sole, Deborah, & Wilson, Daniel Gray. 2002. Storytelling in Organizations: The Power and Traps of Using Stories to Share Knowledge in Organizations Working Paper. Boston, MA: Harvard University.
Swan, J., Scarbrough, H., & Robertson, M. (2002). The construction of 'communities of practice' in the management of innovation. Management Learning, 33(4), 477-496. doi: 10.1177/1350507602334005
Tan, Hai Chen, Carrillo, Pat, Anumba, Chimay, Kamara, John M., Bouchlaghem, Dino, & Udeaja, Chika. (2006). Live Capture and Reuse of Project Knowledge in Construction Organizations. Knowledge Management Research & Practice, 4, 149-161.
Tyler, Jo A. (2007). Incorporating Storytelling into Practice: How HRD Practitioners Foster Strategic Storytelling. Human Resource Development Quarterly, 18(4).
Ward, Maryanne. (2007). Preserving and Using Institutional Memory Through Knowledge Management Practices A Synthesis of Highway Practice (Vol. Synthesis 365). Washington, DC: National Cooperative Highway Research Program (NCHRP).
Wenger, Etienne. (2010). Conceptual Tools for CoPs as Social Learning Systems: Boundaries, Identity, Trajectories, and Participation. In C. Blackmore (Ed.), Social Learning Systems and Communities of Practice. London: Springer-Verlag London Limited.
Wenger, Etienne, McDermott, Richard, & Snyder, William M. (2002). Cultivating Communities of Practice. Boston: Harvard Business School Press.
Wenger, Etienne, White, Nancy, & Smith, John. (2010). Learning in Communities. In U. D. Ehlers & D. Schneckenberg (Eds.), Changing Cultures in Higher Education. Berlin: Springer-Verlag Berlin
Wenger, Etienne, White, Nancy, & Smith, John D. (2009). Digital Habitats: Stewarding Technology for Communities. Portland: CPSquare.
Winsor, Jeromie, Adams, Louis H., McNeil, McNeil, & Ramasubramanian, Laxmi. (2004). Transportation Asset Management Today: Communities of Practice in the Transportation Industry. Transportation Research Record 1885, 88-95.
102

Appendices: Research Protocols
Appendix A. Survey 1: Baseline
Survey Key Instructions are in bold italics. Survey skip logic is noted in italics. Questions are in bold. Answer options are in normal text.
Implementing Communities of Practice in the GDOT Georgia Institute of Technology
Investigators: Dr. Gordon Kingsley, Dr. Eric Boyer, Dr. Juan Rogers, Dr. Janelle Knox-Hayes
Our research team at Georgia Tech has been asked to conduct a study on knowledge-sharing within the Georgia Department of Transportation. Your contribution to a brief survey below will help us to identify patterns of knowledge exchange within your group, and set up our plan for supporting team over the next two years. The goal of our study is to help teams like yours to improve access to knowledge and to share ideas with like-minded professionals. Our findings from the study can help not only your team, but similar teams in other transportation agencies seeking to share knowledge on topics like roundabout designs. The following survey is similar to the survey- template that we have been sharing with other groups within GDOT. Like many other research studies, the risks involved in completing the online survey are not greater than those involved in daily activities associated with using a computer. Your name and any other fact that might point to you will not appear when results of this study are presented or published, nor will it be shared with other staff at GDOT. The expected time for completing this online survey is 5-10 minutes. Additionally, you do not have to be in this study if you don't want to be; you have the right to change your mind and leave the study at any time without giving any reason and without penalty; and any new information that may make you change your mind about being in this study will be given to you. If you have any questions about this study, you can contact the principal investigator, Dr.Gordon Kingsley at (4040)894-0454 or gkingsley@gatech.edu, or Dr. Eric Boyer at (404) 384-7500 or eric.boyer@pubpolicy.gatech.edu. If you have any questions about your rights as a research participant, you may contact: Ms. Kelly Winn, Georgia Institute of Technology Office of Research Integrity Assurance, at (404) 385-2175. Thank you for your time and contributing to this study!
The following questions were administered to participants:
1. How often do you take responsibility for issues related to [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures] on transportation projects that you work on? Every project I work on Most of the projects I work on Some of the projects I work on
103

A few of the projects I work on None of the projects I work on
2. How much responsibility do you typically have for issues related to [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures] on transportation projects? Primary responsibility Some responsibility Little responsibility Other (please specify)____________
3. To what extent do you agree with the following statement? "I view myself as an expert on adapting fundamentals of [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures] on transportation projects that I work on." [ 1 = do not agree, 10 = completely agree]
4. To what extent do you agree with the following statement? "I am strongly interested in building a virtual community to share knowledge around the application of [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures] to transportation projects." [ 1 = do not agree, 10 = completely agree]
5. If you need information on [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures] when working on a transportation project, who do you ask first? (within GDOT) My supervisor Someone else on my immediate team Someone who reports to me Someone outside of my team, but in my office Someone outside of my office, but within my division Someone outside my division Please indicated name of subject matter expert (optional) ____________
6. If you need information on [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures] when working on a transportation project, what sources would you turn to first? (outside GDOT) Someone in a federal agency Other state DOTs Transportation associations (AASHTO, TRB, etc.)
104

7. To what extent do you agree with the following statement? "I seek out information regarding [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures] as a part of a formal process in my work." [ 1 = do not agree, 10 = completely agree]
8. To what extent do you agree with the following statement? "I advise other people on how to work with [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures]." [ 1 = do not agree, 10 = completely agree]
9. How often do you pose questions to other experts for guidance on [GIS OR practical design OR roundabouts/alternative intersections OR environmental procedures]related issues? Less than once per year At least once per year At least a few times per year At least once per month At least once per week
Appendix B. Survey 2: Transaction Costs
Survey Key Instructions are in bold italics. Survey skip logic is noted in italics. Questions are in bold. Answer options are in normal text.
Implementing Communities of Practice in the GDOT Georgia Institute of Technology
Investigators: Dr. Gordon Kingsley, Dr. Eric Boyer, Dr. Juan Rogers, Dr. Janelle Knox-Hayes
As a follow-up to a round of interviews that our research team from Georgia Tech conducted with members of your team earlier this year, we are writing a brief exercise to assess the "costs" of collecting knowledge within your team. The exercise begins with a "sample technical question" related to your discipline, and is followed by a list of questions asking about "how" you derive an answer to this kind of the problem.
The "sample questions" have been designed with leadership in GDOT to be intentionally challenging, in order to generate "searching" for a solution. This survey is not a "test" of your knowledge, but an examination of how people in your area identify the knowledge they need to solve problems. If you would please read through the entire list of questions before beginning the survey, it would help to explain the process related information that we are interested in. Your
105

responses will help our team to develop knowledge solutions for teams like yours in GDOT. The survey is similar to one that we are sharing with other teams in GDOT.
Your name and any other fact that might point to you will not appear when results of this study are presented or published, nor will be shared with other staff at GDOT. The expected time for completing this online survey is 15-20 minutes.
If you have any questions about this study, you can contact: Dr. Eric Boyer at (404) 385-7500 or eric.boyer@pubpolicy.gatech.edu or Dr. Gordon Kingsley at (404) 894-0454 or gkingsley@gatech.edu. If you have any questions about your rights as a research participant, you may contact: Ms. Kelly Winn, Georgia Institute of Technology Office of Research Integrity Assurance, at (404) 385-2175. Thank you for your time, and participation!
1. TECHNICAL QUESTION: Please read through the entire survey before entering your response. We are interested in "how" you answer this question, as well as the answer that you develop. Please choose one question to answer and indicate your answer in the box below: For surveys with options, participants select one option.
Example from Archaeology (Environmental COP): OPTION 1: "Your project area of potential effects is likely to contain Civil War resources, what sources of information will you seek out in order to properly identify and evaluate this resource type?" OPTION 2: "What steps would you take to determine whether or not your archaeological site is worthy of preservation in place?"
2. Did you find your answer somewhere in writing, or through a conversation with another person(s)? Writing Source Conversation with others
3. How many people did you speak with, in determining your answer to this question? 0 1 2 3 4 More than 4
4. Please indicate where the people are located whom you spoke with (check all that apply) Someone on my immediate team Someone outside my team, but in my office Someone from outside of GDOT (other federal or state agency) Someone from a consulting firm Other (please specify) Other __________
106

5. How long did it take you to come up with an answer to this question? 5 minutes or less 5-15 minutes 15-30 minutes More than 30 minutes
6. Can you specify the written sources that helped you answer this question (if any)?
7. Please indicate the order of knowledge "sources" you contacted in solving this question.
First knowledge source that you consulted Written source (document/manual) Written source (website) Person (on my immediate team) Person (outside my team, but in my office) Person (outside of GDOT, federal or state agency) Person (consultant)
Second knowledge source that you consulted Written source (document/manual) Written source (website) Person (on my immediate team) Person (outside my team, but in my office) Person (outside of GDOT, federal or state agency) Person (consultant)
Third knowledge source that you consulted Written source (document/manual) Written source (website) Person (on my immediate team) Person (outside my team, but in my office) Person (outside of GDOT, federal or state agency) Person (consultant)
Other (please specify) Other ___________
107

Appendix C. Survey 3: Post-Test
Survey Key Instructions are in bold italics. Survey skip logic is noted in italics. Questions are in bold. Answer options are in normal text.
Implementing Communities of Practice in the GDOT Georgia Institute of Technology
Investigators: Dr. Gordon Kingsley, Dr. Eric Boyer, Dr. Juan Rogers, Dr. Janelle Knox-Hayes
After over a year of working with your community of practice, we are sending one last survey to better understand the processes of knowledge sharing in GDOT. The following survey is similar to one that we have been sharing with other people at GDOT, and we will share the results from this survey in the early fall. By filling out this survey, you will help us to better understand the roles of communities of practice in GDOT lessons that can help not only GDOT but other transportation departments across the U.S. The expected time to finish the survey is 10 minutes. Thank you for your participation.
Like many other research studies, the risks involved in completing the online survey are not greater than those involved in daily activities associated with using a computer. Your name and any other fact that might point to you will not appear when results of this study are presented or published, nor will be shared with other staff at GDOT. Additionally, you do not have to be in this study if you don't want to be. You have the right to change your mind and leave the study at any time without giving any reason and without penalty and any new information that may make you change your mind about being in this study will be given to you.
If you have any questions about this study, you can contact the principal investigator, Dr. Gordon Kingsley at (404) 894-0454 or gkingsley@gatech.edu, or Dr. Eric Boyer at (404) 3857500 or eric.boyer@pubpolicy.gatech.edu. If you have any questions about your rights as a research participant, you may contact: Ms. Melanie Clark, Georgia Institute of Technology Office of Research Integrity Assurance, at melanie.clark@gtrc.gatech.edu or 404.894.6942.
Thank you for your time, and for contributing to this study!
The following questions were administered to participants: Question no. 3 is about all the 4 COPs. However, Question no. 4 and onwards mention only two COPs: ENV and PDT. Is this accidental or intentional? Make revisions as applicable to this questionnaire.
1. How many years have you worked in GDOT? 1-5 years 6-10 years 11-15 years 16-20 years
108

21-25 years More than 25 years
2. How many years have you worked in the transportation industry? 1-5 years 6-10 years 11-15 years 16-20 years 21-25 years More than 25 years
3. How often do you take responsibility for issues related to [environmental services, practical design issues, GIS, or roundabouts] on transportation projects? Every project I work on Most of the projects I work on Some of the projects I work on A few of the projects I work on None of the projects I work on
4. Who do you contact first within GDOT if you need information related to [environmental services, practical design, GIS, or roundabouts] issues? My supervisor Someone else on my immediate team Someone who reports to me Someone outside of my team, but in my office Someone outside of my office, but within my division
5. How often do you contact the following sources outside of GDOT if you need information related to [environmental services, practical design, GIS, or roundabouts] issues?
Someone from a federal agency At least once per week At least once per month At least a few times per year At least once per year Less than once per year
Other state DOTs At least once per week At least once per month At least a few times per year At least once per year Less than once per year
Transportation associations (AASHTO, etc.) At least once per week At least once per month At least a few times per year
109

At least once per year Less than once per year
Consulting firms At least once per week At least once per month At least a few times per year At least once per year Less than once per year
GDOT-approved peer reviewers At least once per week At least once per month At least a few times per year At least once per year Less than once per year
Retirees from GDOT At least once per week At least once per month At least a few times per year At least once per year Less than once per year
6. How often do you submit questions to other experts for issues related to [environmental services, practical design, GIS, or roundabouts]? At least once per week At least once per month Aat least a few times per year At least once per year Less than once per year
7. How often do you field questions from other people who are seeking guidance on [environmental services, practical design, GIS, or roundabouts]? At least once per week At least once per month At least a few times per year At least once per year Less than once per year
8. What sources do you seek information from most often when you are solving a technical issue related to [environmental services, practical design, GIS, or roundabouts]? Written source (procedures) Written source (other) A software application Another person (SME, supervisor, expert, or other)
110

9. Can you specify the written sources of information that are more helpful to you in solving technical issues related to [environmental services, practical design, GIS, or roundabouts]?
10. Please indicate the order of knowledge "sources" that you typically turn to in solving a technical question related to [environmental services, practical design, GIS, or roundabouts].
First knowledge source that you consulted Written procedure Written (other) A software application Another person
Second knowledge source that you consulted Written procedure Written (other) A software application Another person
Third knowledge source that you consulted Written procedure Written (other) A software application Another person
Other (please specify) ________________
11. How many people do you typically need to speak with when you are solving a challenging technical issue related to [environmental services, practical design, GIS, or roundabouts]? 0 1 2 3 4 More than 4
12. How long does it take you to solve a challenging question related to [environmental services, practical design, GIS, or roundabouts]? Five minutes or less 5-15 minutes 15-30 minutes More than 30 minutes
13. Please rank how useful you find the following knowledge sources/activities for supporting knowledge sharing within your community of practice. [1 = not at all useful, 5 = indifferent, 10= very useful, n/a]
111

A SharePoint site, for sharing documents A SharePoint site, for finding people in the community (by expertise) A SharePoint site, for other reasons In-person trainings, workshops, or meetings to share lessons "Virtual meetings" to share lessons Meetings with other people "informally" for coffee, lunch, etc.
14. To what extent do you agree with the following statements? [1 = do not agree, 10 = strongly agree] I view myself as an expert on [environmental services, practical design, GIS, or roundabouts] I seek information or share information about [environmental services, practical design, GIS, or roundabouts] as part of a formal process that I have to follow in my job I seek information or share information about [environmental services, practical design, GIS, or roundabouts] because I want to (whether or not it's a formal part of my job) I field questions from other members of my community of practice, when they have questions about [environmental services, practical design, GIS, or roundabouts]
15. To what extent do you agree with the following statements?[ 1 = do not agree, 10 = strongly agree] I have a sense of where to locate written sources of knowledge related to [environmental services, practical design, GIS, or roundabouts] I know how to locate other people with expertise related to [environmental services, practical design, GIS, or roundabouts] in my community of practice I am a member of the community of practice on [environmental services, practical design, GIS, or roundabouts] I know who the leader(s) of my community of practice is (are) I know who the other members of my community of practice are I know what the goals of my community of practice are
16. Over the past year, I have seen a reduction in the time that it takes me to identify the knowledge I need for my day-to-day work (such as fast ability to identify the "right person" or "right manual" to consult) [1 = do not agree, 10 = strongly agree]
17. If you answered 6-10 to question no. 16, please indicate what you see as the cause of the change (check the box and/or fill in the box)
My use of my community of practice's SharePoint site (if applicable) My attendance at Community of Practice Workshops (if applicable) My involvement in my community of practice Other (please explain)
112

18. I am able to identify more accurate knowledge related to my work, than I was one year ago. In other words, I feel more confident in the accuracy of the knowledge that I retrieve than I was one year ago. [1 = do not agree, 10 = strongly agree]
19. If you answered 6-10 to question no. 18, please indicate what you see as the cause of the change (check the box and/or fill in the box). My use of my community of practice's SharePoint site (if applicable) My attendance at Community of Practice Workshops (if applicable) My involvement in my community of practice Other (please explain)
20. I can solve more complex problems than I could one year ago. [1 = do not agree, 10 = strongly agree]
21. If you answered 6-10 to question no. 20, please indicate what you see as the cause of this change (check the box and/or fill in the box). My use of my community of practice's SharePoint site (if applicable) My attendance at Community of Practice Workshops (if applicable) My involvement in my community of practice Other (please explain)
22. I am more effective at my job than I was one year ago. [1 = do not agree, 10 = strongly agree]
23. If you answered 6-10 to question no.22, please indicate what you see as the cause of this change (check the box and/or fill in the box). My use of my community of practice's SharePoint site (if applicable) My attendance at Community of Practice Workshops (if applicable) My involvement in my community of practice Other (please explain)
24. When you are trying to find an answer to a problem that you encounter in your dayto-day work, how critical are each of the following sources of knowledge for you to get the solution you need? [1 = do not agree, 10 = strongly agree] Finding the "right" person to talk to, to talk through the issue you are working on (perhaps another SME, someone with special expertise or experience in a given area, etc.) Finding the right manual or written guidance to spell out the steps or procedures that you need to follow Finding the right technology (mapping software, planning software, etc.) to help you think through the issues in an informative way
25. Please score the value of the following sources of knowledge for addressing complex challenges in your work. [1 = not valuable, 5 = indifferent, 10 = very valuable] Other people (supervisors, SME's, etc.) Procedural manuals or other written guidance on standards Laws and/or other legal guidance Contract documents
113

Software that I use to carry out my job Databases that I access to find solutions Explain (optional)
26. How often does your community of practice convene (through a workshop, knowledge exchange, training, meeting, or some other format) to share knowledge with one another and to build community? At least once per week At least once per month At least a few times per year At least once per year Less than once per year
27. Please check the following boxes based on your perceptions of the work of your community of practice (check all that are consistent with your understanding of the community of practice on [environmental services, practical design, GIS, or roundabouts]: I conduct informal interactions between members of my community of practice (communications that are not required by procedures or the organization's reporting structure) I am highly motivated to improve my own professional capabilities to contribute to the area of [environmental services, practical design, GIS, or roundabouts] I share knowledge with other people in this community of practice who are located across GDOT (from across the organizational chart, divisions, etc.) I am aware of our community of practice, and I share an interest in the subject I share information and exchange knowledge through this community of practice more than once per year
28. Is there anything else that you would like to add about how you would like to see this community of practice develop?
Appendix D. Semi-Structured Interview Protocol for Near Retirees
The first part of the questions for this study is designed to gain more background on your personal and professional background. Additionally, this section is intended to explore the nature of knowledge sharing and exchange among GDOT staff working on projects like those addressed in innovative (program) delivery. Your name will not be identified with your comments in any reporting of this study.
Please fill out the following biographical information prior to our interview.
1) Please indicate how long you have worked at GDOT? Less than 1 year 1-5 years 5-10 years 10-15 years 15-20 years
114

More than 20 years
2) Have you ever worked in the private sector before? Yes No
3) If you have worked in the private sector before, please indicate the number of years. [textbox for number]
4) What is your job title? Director Manager Other (please specify): [textbox]
5) Can you explain your role in GDOT?
6) What kinds of projects do you work on?
Interview questions, for when we meet.
1) What are you known for? What are you the "go to" person for?
2) What do only you know how to do?
3) When you return from a vacation, what work is usually waiting for you because no one else knows how to do it?
4) When you have to be away from work, what do you worry about (what work isn't getting done or what work isn't being done well)?
5) Do you serve as a mentor to other staff in the agency? Can you explain? What kinds of advice do you give? What kinds of problems do people come to you with?
6) If mentor others, what kinds of people do you provide advice to? My supervisor Someone else on my immediate team Someone who reports to me Someone outside of my team, but within my office Someone outside of my office, but within my division Someone outside of my division
Can you explain?
7) When you are providing advice to other employees, what kinds of questions do they approach you with? Questions about technical issues on projects
115

Questions about regulatory issues or procedures related to projects Can you explain and/or give some examples? 8) What do you see as the greatest "knowledge-value" that you offer to people who come to you for advice? Knowledge of how to reach technical solutions on project designs Knowledge of how to relate technical solutions to regulatory/procedural issues Knowledge of regulatory/procedural guidelines on projects Knowledge of "who to talk to" on issues related to technical designs Knowledge of "who to talk to" on issues related to regulatory/procedural issues 9) What steps would you carry out if you were going to retire? How would your unique knowledge be documented or preserved for the agency? 10) Is there anything that you would like to add?
116

Appendix E. Workshop Materials
Workshop #1: Knowledge Management Fundamentals
Prior to this workshop, many participants had completed the survey on knowledge management and communities of practice. Therefore, the concepts of knowledge management and communities of practice are not new to you, and you have already begun to consider how an organization can manage knowledge effectively. By managing knowledge and developing communities of practice, organizations can codify knowledge, which will ultimately benefit the organization. Data are presented on the initial findings from the survey to communicate the functions and behaviors of your community of practice.
In the next group exercises, we would like to probe deeper into the characteristics of your communities of practice and begin to model how you can create and capture knowledge.
I. Moderator and record keeping of group activity
Please name a member of the group to lead and moderate the group discussion and another member to keep notes of your discussion and observations.
II. Activity 1: Identifying Characteristics of Your Community of Practice
This first activity is devoted to identifying your community's core characteristics. During this activity, you will form groups to discuss what you see as your community's core characteristics and include the three elements of community of practice: domain, community, and practice.
a. After you've formed a group, identify what your COP's core characteristics are. b. Once everybody in the group is ready, each group will communicate their
understanding of the COP's core characteristics to all workshop attendees; and the attendees discuss their impressions as a whole.
III. Activity 2: Modeling Knowledge Creation/Capture
For this activity, you will form groups and model two situations: storytelling and lessons-learned debriefing. In each situation, you will examine how you initiate conversation, the types of stories that are discussed, and how your group verified the insights from the story.
a. First, begin modeling a situation of story-telling, or an unstructured exchange of knowledge. Consider the following questions: i. How do you initiate conversation? ii. What "type" of story was it? iii. How did your group "verify" the insights of the story, if at all? Through SME, technical principals, regulations?
b. Now, model a situation of a lessons-learned debriefing, or a structured exchange of knowledge. Consider the following questions: i. How do you initiate the conversations?
117

ii. What "type" of story was it? iii. How did your group "verify" the insights of the story, if at all? Through
SME, technical principals, regulations?
IV. Conclusions:
Organize your notes to share with the other groups. Consider the characteristics of a community of practice and the potentials for structured and unstructured knowledge capture.

Workshop #2: Fundamentals of SharePoint Usage in a Community of Practice

This workshop focuses on two key questions: 1) where is your community at? and 2) what could be the next steps for developing the community, and sharing more knowledge? Many people who participated in the knowledge survey indicated that they had never participated in a community of practice; this may mean that many of you have indeed not participated in a community of practice, or you have but use a different term. Communities of practice help to develop members' capabilities through the exchange and building of knowledge; communities of practice are important because not all knowledge can be codified and this provides a platform for that type of knowledge exchange.

In this workshop, I will introduce to you the software, SharePoint, which will help you to participate in a community of practice.

I.

SharePoint Software-Walking you through the site

During this time, I will walk you through the site, its features, and how to use them. You will work individually, as well as in a group to edit the site.
a. First, form a group and then log-on to SharePoint. b. First, we will look at the landing page.
The landing page offers a brief description of the sub-group you belong to. Next, I would like you to consider the following question:
i. Does the brief description you see adequately capture the theme of your sub-group?
ii. Next, take a moment with your group to edit or update the current landing page. Make sure that this is something that reflects your subgroup accurately.
c. Next, we will look at member profiles and practice searching for people in GDOT. There is an option on SharePoint to search for people on the site. We will now try this together. i. First, conduct a search for "intersection" among the people. What did you find? ii. Now, click on your personal profile. Can you add anything in your profile? 1. How could the rest of the members on this site be encouraged to fill this part of the website out?

118

d. Now, we will look at the Document Library. Please consider the following questions: i. How does the document library look? ii. Is there anything that is missing? iii. Try to search for PDP. What comes up? iv. Are there other scopes that need to be included? 1. Note: Searches can be narrowed to any specific SharePoint Site within GDOT.
e. Finally, we will look at the discussion boards and review how to sign up for alerts. Signing up for alerts will ensure that you know when someone posts to a particular discussion board. i. What can be done to help others sign up for alerts? ii. Now, identify 2-3 questions for your group that you might ask on a discussion board. iii. Upload those questions to the discussion board. iv. Respond to 3-4 people somewhere on the board. 1. Are you getting alerts?
II. Conclusions
Communities of practice help to build relationships and to cultivate shared understanding, while also centralizing written knowledge sources. You can help to further develop your community of practice by creating more knowledge-sharing activities and developing virtual platforms for sharing information and knowledge.

Workshop #3: Knowledge Retention Fundamentals

This workshop is the third in a series of workshops that aims to share ideas on how a knowledge-base of expertise on roundabouts can be cultivated within GDOT. Sharing knowledge is not always easily accomplished given that much of the knowledge is stored within individuals and is rarely codified; this tacit knowledge, however, can be transferred to others through storytelling and lessons-learned debriefings. In this study, we are also looking at the role of SharePoint in supporting your community of practice.

The following activities are meant to help you better understand how you can create and record knowledge in your community of practice.

I.

Activity 1--What do you bring to this community, how do you share?

During this activity, you will think through the three stage process of identifying

what you know and how you share it more effectively with others.

a. Stage 1-Knowledge Assessment

i. First, we have to determine our own personal knowledge and then the

critical sources of explicit knowledge.

b. Stage 2--Leveraging External Knowledge

i. Much of the knowledge that is needed is from sources outside of an

organization; determining what sources of knowledge are needed from

the outside is crucial, including tacit knowledge from people and explicit

knowledge from documents.

c. Stage 3--Learning by Doing (Capturing Knowledge)

119

i. Much of the knowledge gained through experience is temporary; therefore, it is important to capture such knowledge. First, you must determine how the group will capture ongoing lessons, including through after-action reporting and/or storytelling.
ii. SharePoint is one means of capturing experiential learning.
II. Activity 2--What are other DOTs doing? a. Many DOTs are facing similar challenges: the need to capture knowledge from an aging workforce that will soon retire, the changing nature of the government workplace, the increasing need to share knowledge with the public, downsizing of the workforce, and hiring freezes. b. One thing other DOTs are doing is assigning roles within the community for people to support explicit and tacit knowledge sharing. Such roles include: leadership roles, knowledge intermediary roles, and community support roles.
III. Activity 3--Practicing Knowledge-Sharing
During this activity, you will form groups and model group-level knowledge transfers. a. Form a group of 4-5 people. b. Next, you will go around the group and each person will bring up 1-2 examples of a challenge s/he encountered on a roundabout and answer the following questions for each example:
i. What happened? ii. What did you try to do to address it? iii. What insights are you looking for from others? c. Next, each person in the group will identify 1-2 successes from a roundabout experience, and answer the following questions for each success: i. What happened? ii. How did you figure it out? iii. What lesson can you share with others? Could this lesson be stored
somewhere?
IV. Activity 4--Practice Expert Interviews (Knowledge Retention)
During this activity, you will interview someone and come up with a plan on how to transfer the knowledge created in the interview.
a. First, pair off with someone. b. Interview this person and then identify what you learned. c. After the interview, work together to formulate a plan to transfer knowledge
and answer the following questions: i. How did your plan come out? ii. What do you think of this kind of process for supporting knowledge transfer?
V. Conclusion
120

In order to build a community of practice, a plan must be created and roles must be assigned to members of the group. Additionally, it is important to understand the nature of your group, adopt a knowledge management process, and choose activities that will help members share their knowledge.
Appendix F. Supplementary Material
Definitions of Potential COP Activities (from Table 13)
Formal Trainings In-person events that include a curriculum and one or more instructors that aim to transfer knowledge from one or more persons to a group.
Formal Knowledge Sharing Activities In-person events organized to share experiences among members of a COP. The events can have very structured approaches to the discussions, like project debriefing guidelines, or may be less structured where members simply discuss their own personal challenges and solutions, like storytelling. In either case, the focus is on memberto-member sharing of lessons and experiences.
Informal Knowledge Sharing Activities These are in-person events with little or no agenda. They are distinguished from formal knowledge sharing activities in that the purpose of meeting is to build relationships primarily, whereas the prior meetings are structured around knowledge sharing. Examples can include meeting for meals.
Listserve Postings Online communication fostered through a Listserve to build community and to share ideas through collective email communication. Assigning a Coordinator for the COP A decision made on the part of leadership in a COP to assign a formal role to a member of the COP to coordinate communication on a virtual forum.
SharePoint Knowledge Sharing Establishing a virtual platform, in this case SharePoint, to coordinate communication across members of a COP.
Centralizing Knowledge Sources Creating a shared source of documentation for members of a COP, where all members can access and update the same source.
Create SME Lists Developing a list of subject matter experts (SMEs) who can be consulted by members of a COP to carry out their work together.
Expert Interviewing Conducting in-depth interviews with employees with particular expertise, in order to translate tacit expertise into explicit expertise and to share knowledge across persons with differing levels of experience.
121