WEBVTT Kind: captions Language: en 1 00:00:00.960 --> 00:00:06.930 ROBERT KARATSU: Okay, good afternoon and we are at our final panel of the day and going 2 00:00:06.930 --> 00:00:11.929 to save the best for last here. And before we get started, I really did want to thank 3 00:00:11.929 --> 00:00:18.050 Maura and the IMLS staff for putting this thing together. You guys did a fantastic job. 4 00:00:18.050 --> 00:00:21.399 (Applause). >> ROBERT KARATSU: And then also to all of 5 00:00:21.399 --> 00:00:27.050 the presenters that shared their inspiration with us today Thank you, all, so much, as 6 00:00:27.050 --> 00:00:31.810 well. And I have actually a little personal thank 7 00:00:31.810 --> 00:00:38.289 you, too, actually again to the IMLS. I mean, John mentioned earlier that going to Washington 8 00:00:38.289 --> 00:00:42.640 D.C. was like the best thing that ever happened to him professionally and I can say the same 9 00:00:42.640 --> 00:00:48.949 thing back in 2013 my library did win the national medal and I did get to go to DC although 10 00:00:48.949 --> 00:00:52.429 John I'm kind of disappointed you're not wearing the medal around your neck. 11 00:00:52.429 --> 00:00:58.230 >> (Away from microphone). >> ROBERT KARATSU: Because I know in rancho 12 00:00:58.230 --> 00:01:03.960 Michelle made a replica for her cat to wear around I think it was a replica, right we 13 00:01:03.960 --> 00:01:11.430 could do a whole program what you can do with a national medal but I digress so anyway our 14 00:01:11.430 --> 00:01:18.320 topic is going to be on evaluation and sustainability. And I thought I would start our part just 15 00:01:18.320 --> 00:01:23.340 to try to frame it maybe with a little story that if you're a library Director or an administrator 16 00:01:23.340 --> 00:01:29.500 or manager and you're in an institution that chases money or in other words like to write 17 00:01:29.500 --> 00:01:34.079 grants well then this may be a little situation that you're going to find yourself in because 18 00:01:34.079 --> 00:01:39.640 this is what I used to do all the time whenever we're in like the visioning or the brainstorming 19 00:01:39.640 --> 00:01:45.689 session, you know, trying to get a grant together there would generally be about four questions 20 00:01:45.689 --> 00:01:51.420 I would ask the person with this great idea, okay, so tell me about your -- what you want 21 00:01:51.420 --> 00:01:56.430 to propose. So the first thing I would ask them, what need is this thing addressing? 22 00:01:56.430 --> 00:02:02.100 Okay. Sounds good. Next one. So tell me about the innovative partnerships. You know, we 23 00:02:02.100 --> 00:02:05.220 all heard about partnerships today. Tell me about these innovative partnerships you've 24 00:02:05.220 --> 00:02:11.520 got for this project. All right, sounds good. Next one is you've got to tell me about measurable 25 00:02:11.520 --> 00:02:17.880 outcomes and outputs. Okay. Sounds good. And then the last one it would always be the 26 00:02:17.880 --> 00:02:25.090 S word, so how are you going to sustain this thing once the grant period is over? And if 27 00:02:25.090 --> 00:02:31.570 they say, I'm going to get another grant, wrong answer And so what we have got today 28 00:02:31.570 --> 00:02:37.390 is our panel, they have kind of figured out that they have unraveled that Gordian Knot 29 00:02:37.390 --> 00:02:41.290 of how to sustained the project and I know they will all be talking about things that 30 00:02:41.290 --> 00:02:48.970 started initially with an LSTA grant or something from the IMLS so really with that I'm going 31 00:02:48.970 --> 00:02:55.270 to turn it over to our panel. We'll introduce them. First we have Dr. Marcia Mardis, she 32 00:02:55.270 --> 00:03:01.250 is a associate Professor at the School of Information, Florida State. Next to her we 33 00:03:01.250 --> 00:03:07.690 have Julie Walker. She is the state librarian from the Georgia Public Library Service. And 34 00:03:07.690 --> 00:03:13.120 then on the end, my former partner in crime, she is the Director of the Rancho Cucamonga 35 00:03:13.120 --> 00:03:17.879 Public Library, Michelle Perera but with that, I give it to you, Marcia, take it away 36 00:03:17.879 --> 00:03:22.910 >> MARCIA MARDIS: I thought we were doing slides today I guess we're not all doing slides 37 00:03:22.910 --> 00:03:29.210 but I'm doing slides so we're going to talk a little bit about the journey from bench 38 00:03:29.210 --> 00:03:37.940 to trench of Web2MARC, which is a spinoff of a 2009 early career researcher grant And 39 00:03:37.940 --> 00:03:50.080 I hope you enjoy it. So let's see. If I can get this go. Web2MARC was born of an idea. 40 00:03:50.080 --> 00:03:57.360 In the last 150 years we've gone from the discovery of germs to organ transplant to 41 00:03:57.360 --> 00:04:04.700 being able to print ears on a 3D printer. We have gone from telegraph to telephone to 42 00:04:04.700 --> 00:04:10.650 more computing power that can put a man on the moon that will put a man on the moon however 43 00:04:10.650 --> 00:04:16.530 you'll notice the classroom hasn't changed all that much in 150 years and one of the 44 00:04:16.530 --> 00:04:21.880 big reasons why it hasn't changed is because content delivery hasn't really changed so 45 00:04:21.880 --> 00:04:27.180 that was really the core of the project that Web2MARC was part of. When we started our 46 00:04:27.180 --> 00:04:31.830 initial research around that project we were out in the field. And talking to teachers 47 00:04:31.830 --> 00:04:36.040 and they said, well, we don't integrate digital content because we don't have the time to 48 00:04:36.040 --> 00:04:40.780 find it and once we find it, frankly, well, I'll translate their personal information 49 00:04:40.780 --> 00:04:46.939 management practices are a little wanting. So my suggestion was why don't you work with 50 00:04:46.939 --> 00:04:51.800 your school librarian and so I refer you back to Renee Franklin's comments just a moment 51 00:04:51.800 --> 00:04:57.229 ago and the school librarian said well I don't know what to do with the stuff either I can 52 00:04:57.229 --> 00:05:01.550 build these hot lists but then I have to keep them up to date I don't have access to the 53 00:05:01.550 --> 00:05:08.320 school Web site yada yada yada I said that destiny system that costs your district $10,000 54 00:05:08.320 --> 00:05:13.840 a year have you thought about cataloging the ones that are currently used that's the other 55 00:05:13.840 --> 00:05:19.790 C word in case you're wondering and they were no longer learning to catalog and the thought 56 00:05:19.790 --> 00:05:24.499 of cataloging something terrified them so I thought this is an interesting problem so 57 00:05:24.499 --> 00:05:31.030 we built a little tool called Web2MARC which you can Google right now Version 1.0 is on 58 00:05:31.030 --> 00:05:37.830 a public Web site and it caught on really quickly so we have tens of thousands of users 59 00:05:37.830 --> 00:05:42.760 overnight so Web2MARC is very simple you paste a URL in a box goes off to the site goes to 60 00:05:42.760 --> 00:05:49.860 content on the site maps it out to a mark record then there are scaffolds to assign 61 00:05:49.860 --> 00:05:54.900 a Common Core standard and next generation science standard and makes educated guesses 62 00:05:54.900 --> 00:06:00.889 through the natural language processing on assigning keywords and you download the mark 63 00:06:00.889 --> 00:06:06.729 records suck it into your online catalog, viola. So I was really surprised that it caught 64 00:06:06.729 --> 00:06:13.270 on this quickly because I would know so through the wonders of graduate students your servers 65 00:06:13.270 --> 00:06:17.740 aren't quite as stable as you would like them to be so I found out because somebody in France 66 00:06:17.740 --> 00:06:21.900 told me before my graduate student even found out that the server was down so it was that 67 00:06:21.900 --> 00:06:27.219 kind of reach that we had really, really quickly. And then the project ended And we thought, 68 00:06:27.219 --> 00:06:31.300 man, how are we going to sustain this thing we've got people who really love it so then 69 00:06:31.300 --> 00:06:34.840 the Office of Commercialization came sniffing around. 70 00:06:34.840 --> 00:06:43.469 And so they talked me into participating in a shark tank which is not -- not the most 71 00:06:43.469 --> 00:06:48.130 pleasant experience you've ever had And if you've ever seen the TV show, it is kind of 72 00:06:48.130 --> 00:06:55.660 like that. Only people aren't as nicely dressed. And so I -- so as you know from watching Shark 73 00:06:55.660 --> 00:07:00.650 Tank there are really two options Ugly Baby or Squatty Potty and if you know Squatty Potty 74 00:07:00.650 --> 00:07:07.180 one of their most successful inventions I was on the ugly baby end of the spectrum but 75 00:07:07.180 --> 00:07:14.919 somehow I got magically back channeled $20,000 to build what we wanted to build to sustain 76 00:07:14.919 --> 00:07:22.270 Web2MARC which is the button on share it toolbar you can pin it on Pinterest tweet it or create 77 00:07:22.270 --> 00:07:27.050 a MARC record for it right from there and download it we get the $15,000 we start to 78 00:07:27.050 --> 00:07:32.110 build the prototype we run into time after time after time we're not getting anywhere. 79 00:07:32.110 --> 00:07:40.889 So we then find that I'll go to this one because -- wow this is not the edited version what 80 00:07:40.889 --> 00:07:46.159 do you know so this is the slide I wanted to show you. 81 00:07:46.159 --> 00:07:50.650 So what I recommend is if your project is something that creates a product and anybody 82 00:07:50.650 --> 00:07:56.900 in your vicinity has had an NSF grant NSF will give you $50,000 to do that market research 83 00:07:56.900 --> 00:08:03.479 through a rather rigorous process called iCore and iCore strips things down to entrepreneurship 84 00:08:03.479 --> 00:08:10.219 I turned to my doctoral student through the first iCore thing I said this is professional 85 00:08:10.219 --> 00:08:15.880 development do you realize what an absolutely beautiful workshop that they are giving us 86 00:08:15.880 --> 00:08:21.030 here about product market fit. About lead with your market. You have to understand what 87 00:08:21.030 --> 00:08:25.089 people want. You don't try to shove your product on them. You have to understand what their 88 00:08:25.089 --> 00:08:30.189 problem really is. And then try to determine whether what you have solves that problem. 89 00:08:30.189 --> 00:08:33.050 Or if what you have could be modified to solve that problem. 90 00:08:33.050 --> 00:08:39.180 So 103 customer interviews later we actually did find out that what we had solved the problem 91 00:08:39.180 --> 00:08:45.300 but not for the end user, for the people who catalog content at repositories like at PBS 92 00:08:45.300 --> 00:08:51.010 at repositories at NSDL and other kinds of content providers, too this is a multi billion 93 00:08:51.010 --> 00:08:58.620 dollar a year industry and bad metadata is costing $958 million alone in the industry 94 00:08:58.620 --> 00:09:10.160 so then the sharks were interested. So I just finished shark tank No. 2 so we're 95 00:09:10.160 --> 00:09:16.470 now in the Squatty Potty realm. So we're getting -- we got into great talks with a company 96 00:09:16.470 --> 00:09:21.190 that I can't name but they used to own a copy machine company and now they own a major education 97 00:09:21.190 --> 00:09:25.290 publisher. That could go well. (Chuckles). 98 00:09:25.290 --> 00:09:31.680 >> MARCIA MARDIS: But to me as I listened to everybody talking today Malcolm Gladwell 99 00:09:31.680 --> 00:09:36.220 was invoked all over the place today and he was invoked when I was writing these slides 100 00:09:36.220 --> 00:09:41.350 as well because really it comes down to the three things everybody is talking about, connection, 101 00:09:41.350 --> 00:09:48.270 making those networks, being expert in being able to I assert your expertise and also being 102 00:09:48.270 --> 00:09:53.800 although salesman is the word that Malcolm had to use but more being the advocate being 103 00:09:53.800 --> 00:09:58.680 out there taking your services to where they need to go then I did a little proof during 104 00:09:58.680 --> 00:10:06.570 the Kansas City event. And so I think what I figured out is what we do is the bench. 105 00:10:06.570 --> 00:10:10.990 Where things need to go in the university realm is into the trench which is the lower 106 00:10:10.990 --> 00:10:17.440 level. So if you want to maintain your project and you're in the trench either you need to 107 00:10:17.440 --> 00:10:22.760 maintain your practice end of things or your research end of things. And you can do that 108 00:10:22.760 --> 00:10:27.740 by scaling out like Project ENABLE did or you can do it by taking it into completely 109 00:10:27.740 --> 00:10:32.790 different direction by building on that thing that's really working like we did in Web2MARC, 110 00:10:32.790 --> 00:10:38.260 it wasn't the public Web site that's working so much as the idea of better metadata and 111 00:10:38.260 --> 00:10:42.910 that's what we're building on so we're exponentially sustaining our project and that's in a different 112 00:10:42.910 --> 00:10:46.120 way so really it's mathematics, everybody. (Chuckles). 113 00:10:46.120 --> 00:10:51.680 >> MARCIA MARDIS: So that's sort of my reflection on that. And if you want to know anything 114 00:10:51.680 --> 00:10:57.260 more about my feelings about bench to trench because I have this whole rant on who is trench 115 00:10:57.260 --> 00:11:04.110 that we can go to. But thanks very much. (Applause). 116 00:11:04.110 --> 00:11:10.490 >> JULIE WALKER: Good afternoon I'm Julie Walker from the Georgia Public Library Service 117 00:11:10.490 --> 00:11:15.420 and I think GPLS has the somewhat deserved reputation of leaping in where angling fear 118 00:11:15.420 --> 00:11:19.430 to trend sometimes because I can remember 12 years ago when we started talking about 119 00:11:19.430 --> 00:11:26.060 building an Open Source ILS and the looks I got like really. But in the aftermath of 120 00:11:26.060 --> 00:11:29.980 that decision we have talked a lot about taking risks and how librarians are usually fairly 121 00:11:29.980 --> 00:11:35.250 risk averse and metaphorically jumping off that cliff but really we did put a tremendous 122 00:11:35.250 --> 00:11:40.240 amount of thought into the sustainability and scalability of the project and really 123 00:11:40.240 --> 00:11:45.780 focused on the long-term results. We really weren't just looking for something cool to 124 00:11:45.780 --> 00:11:49.390 do. We had to -- we of course defined our basic 125 00:11:49.390 --> 00:11:53.420 need and our basic need was staring us right in the face we were absolutely desperate we 126 00:11:53.420 --> 00:11:58.570 were patching together an off the shelf ILS that didn't work for a statewide consortium 127 00:11:58.570 --> 00:12:01.830 and every time they would do an upgrade all of our patches would fail and we would start 128 00:12:01.830 --> 00:12:07.030 all over again and we got to the point where we really had no choice. 129 00:12:07.030 --> 00:12:11.410 We do try to use a lot of the basic tenets of project management when we make decisions 130 00:12:11.410 --> 00:12:16.610 about this and we looked at the buy versus build idea. When we were looking at this. 131 00:12:16.610 --> 00:12:21.580 But there was nothing to buy. We had really done a thorough environmental scan and truly 132 00:12:21.580 --> 00:12:27.720 there was not anything on the marketplace that was designed and built to support a consortium 133 00:12:27.720 --> 00:12:34.050 like the Georgia PINES consortium was built so we absolutely had to do something. 134 00:12:34.050 --> 00:12:41.120 And we decided that we had to build. We had a unique need. It was going to be less expensive 135 00:12:41.120 --> 00:12:43.790 -- well, it wasn't less expensive than to buy -- it was less expensive we could not 136 00:12:43.790 --> 00:12:51.160 buy it and we were the only ones willing to take on that task to accomplish it. 137 00:12:51.160 --> 00:12:55.940 We took a huge amount of time to build the support we needed to sustain this project. 138 00:12:55.940 --> 00:13:01.490 Even though we were kind of going it alone. But our parent agency which is University 139 00:13:01.490 --> 00:13:06.470 of System of Georgia was very skeptical of Open Source software at that time. 140 00:13:06.470 --> 00:13:10.840 And we really had to call in every chip we had just to get them on board but we also 141 00:13:10.840 --> 00:13:16.070 had to work very hard every step of the way to earn their trust and to be really transparent 142 00:13:16.070 --> 00:13:21.720 and have tremendously good communication. So that they would see that we really could 143 00:13:21.720 --> 00:13:25.950 handle this major project And our first goal was just a product that 144 00:13:25.950 --> 00:13:30.100 would work for PINES, work for our member libraries and we were committed to sustaining 145 00:13:30.100 --> 00:13:34.700 it for Georgia and we didn't think so much about planting it around the world at that 146 00:13:34.700 --> 00:13:37.490 time. So what we really needed to do first was secure 147 00:13:37.490 --> 00:13:43.140 the buy-in from our member libraries in PINES Our constituents. So we spent months and months 148 00:13:43.140 --> 00:13:49.440 and months doing focus groups to determine the features they wanted. To gather requirements. 149 00:13:49.440 --> 00:13:54.720 And we wanted mostly to make sure there was a shared ownership before we got to that critical 150 00:13:54.720 --> 00:13:58.360 go-live date. And the Open Source model in that case really 151 00:13:58.360 --> 00:14:02.740 worked in our favor. We were really transparent. There was a huge amount of communication. 152 00:14:02.740 --> 00:14:07.470 The Open Source community is really known for their great communication on any number 153 00:14:07.470 --> 00:14:10.790 of channels. So that helped us. It may have hurt us it 154 00:14:10.790 --> 00:14:15.230 may have been a tactical error in the very first day we went live every Open Source person 155 00:14:15.230 --> 00:14:20.590 in the known world logged in to look at what was happening what's going on in PINES today 156 00:14:20.590 --> 00:14:22.940 and it all crashed. (Chuckles). 157 00:14:22.940 --> 00:14:27.110 >> JULIE WALKER: But the support we received from that Open Source software community was 158 00:14:27.110 --> 00:14:33.830 just invaluable and the proof now is that it's taking on a life of its own. We were 159 00:14:33.830 --> 00:14:38.990 able -- once this worked for PINES and still works for PINES today is a great thing for 160 00:14:38.990 --> 00:14:43.290 us. We're not looking for a different solution. We're just continuing to grow evergreen but 161 00:14:43.290 --> 00:14:47.280 it's way beyond the borders of Georgia and way beyond the borders of the United States 162 00:14:47.280 --> 00:14:54.910 now so that product has really been shown to grow and thrive and we're still very proud 163 00:14:54.910 --> 00:14:59.160 that it was implanted in Georgia for the first time and we still work really hard to continue 164 00:14:59.160 --> 00:15:04.720 to build that evergreen community. We send our entire PINES staff to the evergreen meetings 165 00:15:04.720 --> 00:15:09.570 and they serve on all of the crucial committees for Evergreen so we are absolutely committed 166 00:15:09.570 --> 00:15:14.410 to committing to sustain that product throughout the world and it's really gratifying to see 167 00:15:14.410 --> 00:15:17.850 all of the different libraries that it works for. 168 00:15:17.850 --> 00:15:23.660 But once we caught our breath from Evergreen, a new challenge popped up And the idea started 169 00:15:23.660 --> 00:15:29.110 to build that, hey, we could take the basis of the Evergreen software and build a product 170 00:15:29.110 --> 00:15:33.170 for the libraries for the blind and physically handicapped because we were hearing from a 171 00:15:33.170 --> 00:15:39.100 lot of state libraries that the one product that was available for them was somewhat unsatisfactory 172 00:15:39.100 --> 00:15:46.060 and we said let's take a look at that so thanks to a grant an IMLS grant we were able to pull 173 00:15:46.060 --> 00:15:51.390 together a convening in Atlanta with people from the large majority of the state libraries 174 00:15:51.390 --> 00:15:59.010 to come and talk about that idea and to do a requirements gathering and it was a great 175 00:15:59.010 --> 00:16:04.040 robust useful week-long conference The correct participants were at the table. There was 176 00:16:04.040 --> 00:16:10.180 a lot of enthusiasm about the project. But it was really a case of poor timing because 177 00:16:10.180 --> 00:16:15.410 we had a great White Paper that came out of the project and then we started talking 178 00:16:15.410 --> 00:16:18.790 to the people who had come and who had been the most enthusiastic about participating 179 00:16:18.790 --> 00:16:26.970 about who is in, who is in for funding and helping us and providing resources to build 180 00:16:26.970 --> 00:16:33.340 this. And that was right at the moment the economy went way, way, way south and nobody 181 00:16:33.340 --> 00:16:38.050 had the bandwidth, nobody had the capacity, nobody had the resources to do that project 182 00:16:38.050 --> 00:16:41.990 at that time. We couldn't undertake it alone and nobody 183 00:16:41.990 --> 00:16:46.940 else was able to step up to partner with us. So we had to learn when to let it go and I 184 00:16:46.940 --> 00:16:51.380 think that's a really valuable lesson when we're talking about grant funded projects 185 00:16:51.380 --> 00:16:56.430 or any of these new innovative ideas. Sometimes the time is just not right. Sometimes you 186 00:16:56.430 --> 00:17:01.600 can't afford to let yourself get so attached to an idea that it's going to drain the resources 187 00:17:01.600 --> 00:17:04.569 away from the other things that you have to be doing. 188 00:17:04.569 --> 00:17:09.750 So Loblolly, which was the name of that project Mayet resurface and it's a great example of 189 00:17:09.750 --> 00:17:16.699 knowing when to is step back from an unsustainable idea quickly I'll talk about the project we're 190 00:17:16.699 --> 00:17:24.039 through it's the B4 early Literacy Initiative another IMLS project birth to four or years 191 00:17:24.039 --> 00:17:31.950 before kindergarten focuses on family literacy and reading books before kindergarten so our 192 00:17:31.950 --> 00:17:36.870 libraries are really running with that this summer The great thing about this project 193 00:17:36.870 --> 00:17:40.799 and the thing we're really excited about is that we looked at existing models. We didn't 194 00:17:40.799 --> 00:17:46.630 try to develop a new thing. Just to say, GPLS had developed a new thing we took two nationally 195 00:17:46.630 --> 00:17:51.830 recognized early literacy programs Every Child Ready to ready and Prime Time Preschool very 196 00:17:51.830 --> 00:17:58.769 solid and proven programs we're piloting each of those in six different library systems. 197 00:17:58.769 --> 00:18:04.340 And we're really focusing on evaluation. We know that we are not experts on evaluating 198 00:18:04.340 --> 00:18:09.249 early literacy programs. Most librarians aren't. We don't have that background and training 199 00:18:09.249 --> 00:18:13.659 so we have an expert from the University of Georgia She's been involved every step of 200 00:18:13.659 --> 00:18:19.190 the way. She's going to evaluate the participants in those programs and we're going to come 201 00:18:19.190 --> 00:18:24.700 out at the end with some idea of which one is the most effective for families in Georgia. 202 00:18:24.700 --> 00:18:29.010 And then that will allow us once we look at those results to see what we want to roll 203 00:18:29.010 --> 00:18:34.009 out statewide so those are three big things that we've been working on in Georgia that 204 00:18:34.009 --> 00:18:41.990 our wonderful Federal partners have funded for us and some very different outcomes. Thanks. 205 00:18:41.990 --> 00:18:47.409 (Applause). >> MICHELLE PERERA: Good afternoon everyone 206 00:18:47.409 --> 00:18:51.779 I'm Michelle Perera I'm from the Rancho Cucamonga library and I want to talk to you about our 207 00:18:51.779 --> 00:18:55.610 Play and Learn Islands which is probably my favorite project I've worked on or at least 208 00:18:55.610 --> 00:19:01.769 one of two favorite projects and just in luck I'm going to mention both of them so our play 209 00:19:01.769 --> 00:19:05.860 started with a needs assessment we were trying to figure out with what to do with 14,000 210 00:19:05.860 --> 00:19:10.970 square feet we have on our second floor and it was overwhelmingly a response from our 211 00:19:10.970 --> 00:19:15.549 public that they wanted some kind of interactive experience for their kids. They wanted their 212 00:19:15.549 --> 00:19:19.750 children to learn something through play. They wanted their kids to be able to come 213 00:19:19.750 --> 00:19:24.679 to the library and do something and it was right when the economy tanked so we didn't 214 00:19:24.679 --> 00:19:28.999 have the money to complete the second floor but we thought we can still do something with 215 00:19:28.999 --> 00:19:34.450 this data so we decided to create our Play and Learn Islands which are small scale interactive 216 00:19:34.450 --> 00:19:38.659 exhibits for children we were fortunate to get several grants from the California State 217 00:19:38.659 --> 00:19:45.080 Library to help us fund it as well as some money from our library foundation so we contracted 218 00:19:45.080 --> 00:19:50.379 with a museum designer because those are the experts on creating exhibits. We just got 219 00:19:50.379 --> 00:19:58.210 a couple of pictures that are behind me to show you one was about building kids learning 220 00:19:58.210 --> 00:20:04.299 about design, redesigning another one about digging really very versatile it could be 221 00:20:04.299 --> 00:20:09.929 a sandbox with a dynamo big or Chinese New Year program we put tare cot at that warriors 222 00:20:09.929 --> 00:20:14.740 in there they got to dig those out we found it was something you could use for any type 223 00:20:14.740 --> 00:20:20.440 of event. We also created a Farmer's Market. A little stand. It's a miniature Farmer's 224 00:20:20.440 --> 00:20:25.240 Market that really worked well with our healthy communities initiative in Rancho Cucamonga 225 00:20:25.240 --> 00:20:30.919 and has been popular in other libraries because a lot of cities are doing healthy communities 226 00:20:30.919 --> 00:20:37.769 initiatives and it really conveys the concept of from the field to the market to the table. 227 00:20:37.769 --> 00:20:41.840 Actually there's the one right there with just some pictures of the vegetables and there's 228 00:20:41.840 --> 00:20:47.149 mulch in there they can actually dig and find what they are looking for. 229 00:20:47.149 --> 00:20:52.101 So we piloted these in our libraries. And we found them to be pretty darn popular. Kids 230 00:20:52.101 --> 00:20:56.840 -- we were worried about having to teach kids what to do with them and how to use them. 231 00:20:56.840 --> 00:21:00.059 Kids don't need to be taught anything. They just kind of roll up to something that looks 232 00:21:00.059 --> 00:21:05.019 fun and they play, they learn about sharing and really those good socialization skills 233 00:21:05.019 --> 00:21:08.940 with other kids, as well. So it worked in our library and we thought 234 00:21:08.940 --> 00:21:13.419 well if it can work in our library maybe it can work in other libraries so we went back 235 00:21:13.419 --> 00:21:17.600 to the State Library and applied for some more money to build some more of these that 236 00:21:17.600 --> 00:21:24.190 we could loan for free to other California libraries. And they were super popular in 237 00:21:24.190 --> 00:21:28.220 the other libraries. Well, they work for us, why not work for others. Because the great 238 00:21:28.220 --> 00:21:33.510 thing about them was it gave people something to do when you walked into that library. 239 00:21:33.510 --> 00:21:37.400 There was always an activity out there. It could be the exact same activity every single 240 00:21:37.400 --> 00:21:42.389 time that child walks in because some kids like repetition or it could be completely 241 00:21:42.389 --> 00:21:46.450 different every time you walked in. But the best part of it was it was unstructured and 242 00:21:46.450 --> 00:21:51.140 they could make it whatever they wanted. So we started loaning them to libraries and 243 00:21:51.140 --> 00:21:53.259 we found libraries didn't want to give them back. 244 00:21:53.259 --> 00:21:56.120 (Chuckles). >> MICHELLE PERERA: So you get the repo truck 245 00:21:56.120 --> 00:22:00.460 to go out there -- no. Is so we thought wait a minute why aren't we selling these to other 246 00:22:00.460 --> 00:22:05.250 libraries so we created a plan to try to sell these to libraries. We loaned them to about 247 00:22:05.250 --> 00:22:11.880 35 libraries. We have since sold them to about 40. And it has just been a great experience. 248 00:22:11.880 --> 00:22:16.629 I mean imagine being the nerdy librarian that I am wherever I am I'm going to pop into the 249 00:22:16.629 --> 00:22:22.710 local library and we were in Fulsome for some reason we went no -- into the library and 250 00:22:22.710 --> 00:22:30.201 there was a play and learn library I was walking around looking like a creepy adult in the 251 00:22:30.201 --> 00:22:35.440 childrens room and the librarian said let me tell you about that I thought I created 252 00:22:35.440 --> 00:22:42.259 that and it was cool. I saw a -- I went to another library and say a play and learn library 253 00:22:42.259 --> 00:22:50.830 and ripped us off and copied our design. But as part of the process for creating these 254 00:22:50.830 --> 00:22:56.610 we also trademarked them. Really came up with a good plan in order to try to sell them to 255 00:22:56.610 --> 00:23:03.019 other libraries. So that -- having a little bit of money that 256 00:23:03.019 --> 00:23:08.600 we earned from these helps us to sustain this project. It allows us to create more for our 257 00:23:08.600 --> 00:23:12.419 library, it allowed us to design a few more but that's not really what's going to sustain 258 00:23:12.419 --> 00:23:17.840 the project money as hard as it is to find isn't the hardest thing to find in the world. 259 00:23:17.840 --> 00:23:23.570 What sustaining it are staff so we really spent a lot of time training our staff in 260 00:23:23.570 --> 00:23:28.539 early childhood development and brain development and that cognitive learning skills because 261 00:23:28.539 --> 00:23:33.610 that's what's going to sustain it they are excited about it they feel confident about 262 00:23:33.610 --> 00:23:38.450 this, they want to create more so by giving them the knowledge and not just saying we're 263 00:23:38.450 --> 00:23:43.529 going to embark on a project and not give you any training or resources which is what 264 00:23:43.529 --> 00:23:49.919 we often do a little too often, that really got them excited and bought into the project 265 00:23:49.919 --> 00:23:56.970 and what's going to keep it going so now I'm going to segue to our staff innovation fund 266 00:23:56.970 --> 00:24:03.039 which is also about training and sustainability because we realized well if we train them 267 00:24:03.039 --> 00:24:09.210 on this area early learning, and they are keeping this project alive very easily what 268 00:24:09.210 --> 00:24:15.789 if we expanded that so we created a ten part training program, grants public speaking, 269 00:24:15.789 --> 00:24:21.299 marketing and branding, dealing with conflict, putting together teams, trend watching and 270 00:24:21.299 --> 00:24:27.900 by putting that training series together, our staff really was able to gain a lot of 271 00:24:27.900 --> 00:24:33.570 confidence and be able to sustain any project they wanted to do because they had the skills 272 00:24:33.570 --> 00:24:37.470 to do it so now they are out there trying to find money, they are out there looking 273 00:24:37.470 --> 00:24:43.240 for experts and looking for resources it's not beholden upon me or one person or two 274 00:24:43.240 --> 00:24:49.200 people in my organization to do that so the staff innovation fund, first part was ten 275 00:24:49.200 --> 00:24:53.029 training sessions. The second part is an innovation fund where 276 00:24:53.029 --> 00:24:57.529 they can apply for money to try to create the projects they want and some of them have 277 00:24:57.529 --> 00:25:04.539 gone onto create projects that deal with the Play and Learn Islands one created a renewable 278 00:25:04.539 --> 00:25:10.380 Play and Learn Islands which has a working solar panel, teaches kids about wind energy, 279 00:25:10.380 --> 00:25:16.789 about water, and it -- it's just a great thing to kind of create something and because staff 280 00:25:16.789 --> 00:25:22.520 is trained and they feel confident, they sustain it and they go off and do it themselves. Our 281 00:25:22.520 --> 00:25:28.860 staff innovation fund is something that's been funded by the California State Library 282 00:25:28.860 --> 00:25:33.550 and they have continued to fund it over the years not just for our library but they have 283 00:25:33.550 --> 00:25:37.419 continued to fund it for other libraries in California and to date about 25 libraries 284 00:25:37.419 --> 00:25:43.679 in California have gone through this program, done the ten-part training and also have been 285 00:25:43.679 --> 00:25:49.480 able to apply for money to do the passion projects that really benefit the community 286 00:25:49.480 --> 00:25:54.620 in their own libraries. Thank you. (Applause). 287 00:25:54.620 --> 00:26:00.500 >> ROBERT KARATSU: Okay. So if there's anyone out in the livestream world or in the room 288 00:26:00.500 --> 00:26:05.389 that has questions, comments for the panel and since I have the microphone I've got one 289 00:26:05.389 --> 00:26:10.179 because you all make these things sound so easy but I suspect along the way you ran into 290 00:26:10.179 --> 00:26:14.269 a few road bumps so would you mind sharing some of those things with us. 291 00:26:14.269 --> 00:26:19.679 >> MICHELLE PERERA: I would be happy to share one. I did make it sound great those Play 292 00:26:19.679 --> 00:26:24.619 and Learn Islands came out as well as they did but one of them which is the one we don't 293 00:26:24.619 --> 00:26:31.809 talk about totally sucked. We got this. We thought we looked at the schematics really 294 00:26:31.809 --> 00:26:37.269 well and then it showed up and instead of being a really versatile Play and Learn Islands 295 00:26:37.269 --> 00:26:43.970 has German wheels can turn things easily move around the library go outside this thing weighed 296 00:26:43.970 --> 00:26:50.509 150 pounds we couldn't move it. There were no wheels and it had this -- it was supposed 297 00:26:50.509 --> 00:26:54.039 to be about movement. Yeah, I know. (Chuckles). 298 00:26:54.039 --> 00:26:59.110 >> MICHELLE PERERA: Ironic and it had this pulley that you crank. Well the first crank 299 00:26:59.110 --> 00:27:05.990 we did, it died and the top crank looked like a guillotine for something for kids having 300 00:27:05.990 --> 00:27:11.460 something that looks like a guillotine can chop off their heads not so great and then 301 00:27:11.460 --> 00:27:17.030 we had a book wheel the idea was to look like a ferris wheel of books going around and we 302 00:27:17.030 --> 00:27:21.700 saw this thing and it was taller than we were and kids were trying to climb into it because 303 00:27:21.700 --> 00:27:28.779 they wanted to go around But the best part about this was the ball run that we had on 304 00:27:28.779 --> 00:27:33.169 here which you connect the pipes together and you drop the ball and it goes through 305 00:27:33.169 --> 00:27:42.750 was a choking hazard. So that was a big failure but what we learned from that well, one, learn 306 00:27:42.750 --> 00:27:49.740 what a choking hazard is early and often but then really it's okay if something doesn't 307 00:27:49.740 --> 00:27:56.470 work out because all of the rest were really successful and as we moved forward we learned 308 00:27:56.470 --> 00:27:59.940 we need to have something that moves easily we need to have something that's versatile 309 00:27:59.940 --> 00:28:05.590 we need to have something that kids are going to enjoy. But still be very safe, as well. 310 00:28:05.590 --> 00:28:10.609 So it was a good learning experience. >> ROBERT KARATSU: Okay. Anyone else have 311 00:28:10.609 --> 00:28:16.169 anything people might find interesting that maybe didn't work out the way you thought 312 00:28:16.169 --> 00:28:21.010 it was going to? >> JULIE WALKER: Well we certainly did have 313 00:28:21.010 --> 00:28:25.899 to do a lot of talking to our libraries out in Georgia when we were working on our Evergreen 314 00:28:25.899 --> 00:28:29.120 project because they kind of got to the point of what are those people going to come up 315 00:28:29.120 --> 00:28:35.359 with next and do we really trust that these four guys sitting in the office in Atlanta 316 00:28:35.359 --> 00:28:39.190 drinking Mellow Yellow and eating Snickers bars are going to come up with an ILS that 317 00:28:39.190 --> 00:28:45.609 we can actually trust and use with all of our data and as I said a couple of the things, 318 00:28:45.609 --> 00:28:50.009 one was that everybody in the world looked in on the very first day but also we had a 319 00:28:50.009 --> 00:28:54.679 lot of brand-new staff the day we went live and we had worked because all of the PINES 320 00:28:54.679 --> 00:29:00.489 libraries had to go live the same day we couldn't pilot this. This was an all or nothing proposition. 321 00:29:00.489 --> 00:29:08.409 So our entire staff had been up for four straight days to bring -- migrate all of the patrons, 322 00:29:08.409 --> 00:29:12.669 all of the items all of the transactions, everything to go live the day after Labor 323 00:29:12.669 --> 00:29:18.489 Day. So we were utterly exhausted that first day before it even started so that was probably 324 00:29:18.489 --> 00:29:24.669 not the best idea in the world but we learned to keep our partners and our friends very 325 00:29:24.669 --> 00:29:30.320 close to us to not spring anything on them, to not make any sudden moves they weren't 326 00:29:30.320 --> 00:29:35.799 prepared for so they really felt confident I guess the only other thing that was a little 327 00:29:35.799 --> 00:29:40.090 bit dicey and still to this day there are vendors that I feel like snarl at us once 328 00:29:40.090 --> 00:29:44.919 we walk by because once you have displaced a major vendor product you are never going 329 00:29:44.919 --> 00:29:47.779 to be their best friend anymore so we have learned that. 330 00:29:47.779 --> 00:29:54.789 >> ROBERT KARATSU: Do we have in the room or did anything come in over Twitter? 331 00:29:54.789 --> 00:30:05.999 >> Michelle, are the training programs, are they online or are this face-to-face or how 332 00:30:05.999 --> 00:30:09.190 do you deliver those segments? >> MICHELLE PERERA: They are face-to-face 333 00:30:09.190 --> 00:30:14.179 what we found by having a large number of our staff go through it together or in these 334 00:30:14.179 --> 00:30:21.809 other libraries a large number of staff go through it up to about 35, 40. Is it works 335 00:30:21.809 --> 00:30:27.860 -- it's working -- it works as a good team building experience activity the interaction 336 00:30:27.860 --> 00:30:32.690 between a adult person a childrens person a teen person the library Director and Assistant 337 00:30:32.690 --> 00:30:37.919 Director everybody on the same level of that training was really invaluable and having 338 00:30:37.919 --> 00:30:44.070 the Director and Assistant Director sit through everything really is something that shows 339 00:30:44.070 --> 00:30:48.820 and sets a good example for staff, as well. And they can bounce ideas off and we found 340 00:30:48.820 --> 00:30:53.220 that as a result of that, you're not going to go through the training and the next day 341 00:30:53.220 --> 00:30:57.379 just be more innovative but by going through the trainings and be able to talk and bounce 342 00:30:57.379 --> 00:31:02.070 ideas off of each other you do become more innovative. 343 00:31:02.070 --> 00:31:08.879 >> Hi, Christy with OCLC I love hearing these stories of a sustainable -- of sustainable 344 00:31:08.879 --> 00:31:16.019 projects. I wonder if you could, any of you, talk about how your evaluation approach, either 345 00:31:16.019 --> 00:31:22.830 hindered or helped your ability to become sustainable once the initial startup funding 346 00:31:22.830 --> 00:31:29.029 was spent. >> MARCIA MARDIS: Well, I would say in our 347 00:31:29.029 --> 00:31:36.879 case, our evaluation approach really helped us kind of keep the faith that we were onto 348 00:31:36.879 --> 00:31:42.320 something. And just what form that something needed to take was something that we had to 349 00:31:42.320 --> 00:31:47.190 kind of work through. Because sometimes I think when there's money on the table and 350 00:31:47.190 --> 00:31:51.139 you're asking people for money and they need to invest in you particularly from the private 351 00:31:51.139 --> 00:31:58.010 sector, they are very, very, very skeptical. And you know first question that they want 352 00:31:58.010 --> 00:32:03.970 to know is so how are you going to make me money? And so there were times where we did 353 00:32:03.970 --> 00:32:10.850 have various existential crisis and in fact I'm still going through one now between the 354 00:32:10.850 --> 00:32:15.289 Open Source route and going a commercial route. I mean that's something that I wrestle with 355 00:32:15.289 --> 00:32:22.999 every day. But a good evaluation and good ongoing dialogue with your evaluator is a 356 00:32:22.999 --> 00:32:29.440 way that you can just really keep the faith in your idea I think from my perspective that's 357 00:32:29.440 --> 00:32:32.120 important. >> MICHELLE PERERA: For us with the Play and 358 00:32:32.120 --> 00:32:36.600 Learn Islands project really looking back at how we created them, what we did, what 359 00:32:36.600 --> 00:32:41.929 we needed to tweet what was response was from the public certainly looking at the whole 360 00:32:41.929 --> 00:32:49.251 debacle with the movement one, that really helped us fine tune the design and fine tune 361 00:32:49.251 --> 00:32:54.119 what we wanted to do and as a result of that fine tuning actually brought the price down 362 00:32:54.119 --> 00:33:01.690 about 35%. So we just seemed to have more stuff because when you first do something, 363 00:33:01.690 --> 00:33:04.100 it's like going to the grocery store when you're hungry you just throw everything in 364 00:33:04.100 --> 00:33:07.570 the cart that's what it was like because we were so excited but then if you make that 365 00:33:07.570 --> 00:33:13.720 shopping list and really figure out what works you're kind of able to make it as efficient 366 00:33:13.720 --> 00:33:20.210 and as cost effective as possible. >> JULIE WALKER: And I think the evaluation 367 00:33:20.210 --> 00:33:25.769 effort we're undertaking on our B4 project will be important because we need to show 368 00:33:25.769 --> 00:33:30.769 our libraries that this project really does yield results or maybe we'll show it doesn't 369 00:33:30.769 --> 00:33:34.840 and it's not something that we'll pursue. But if we're asking them to do something new 370 00:33:34.840 --> 00:33:39.000 and different, I mean they are stretched to the limit. They don't have any extra staff 371 00:33:39.000 --> 00:33:42.759 resources, they don't have any extra money, they are overworked and while they are very 372 00:33:42.759 --> 00:33:48.220 game to always try new things I think at this point we really need to prove to them that 373 00:33:48.220 --> 00:33:52.289 something is going to be effective before we ask every library in the state to participate 374 00:33:52.289 --> 00:33:57.289 in it so I think that's going to be the important thing about the evaluation piece of this project 375 00:33:57.289 --> 00:34:01.429 is what do we get at the end and then what can we tell our libraries that will make them 376 00:34:01.429 --> 00:34:05.100 believers. >> MARCIA MARDIS: If I could tag one thing 377 00:34:05.100 --> 00:34:12.240 on to the end there I'm East Coast time I'm starting to wind down. Something that I do 378 00:34:12.240 --> 00:34:16.599 think was part of the evaluation so there's the Web2MARC IMLS grant and the evaluation 379 00:34:16.599 --> 00:34:22.260 that that -- the role that evaluation played in helping us kind of keep the faith but then 380 00:34:22.260 --> 00:34:28.940 in iCore when we had to do 103 customer discovery encounters in four weeks and we were talking 381 00:34:28.940 --> 00:34:34.299 to everybody, their mothers, brothers, cousins, everything we talked to people from Flickr, 382 00:34:34.299 --> 00:34:41.270 Google, enterprise companies, everybody, to be able to actually listen to what people 383 00:34:41.270 --> 00:34:47.030 are telling you and one of the things with iCore you cannot tell them what your technology 384 00:34:47.030 --> 00:34:53.419 is. You just have to ask them so in this sort of situation like with information where do 385 00:34:53.419 --> 00:34:58.460 you find your information, how do you organize it? What's difficult for you? What's easy 386 00:34:58.460 --> 00:35:02.410 for you? If you had to solve your problem, how would you solve that problem? And just 387 00:35:02.410 --> 00:35:07.490 kind of being quiet and listening and not being tempted to say, oh, we have the tool 388 00:35:07.490 --> 00:35:15.059 for you. And really listening that's important, too and as a professor I don't have access 389 00:35:15.059 --> 00:35:21.650 to large groups of people on a day-to-day basis you would think so but in an online 390 00:35:21.650 --> 00:35:26.920 teaching environment that's not necessarily the case. And it was just really important 391 00:35:26.920 --> 00:35:32.500 so that's a luxury all of you have the fact you can walk out and come face-to-face with 392 00:35:32.500 --> 00:35:37.859 the people who make best use of your services I think is probably the best source of ongoing 393 00:35:37.859 --> 00:35:42.650 evaluation you can have. Because you really miss it when it's not there. 394 00:35:42.650 --> 00:35:49.190 >> ROBERT KARATSU: Okay I don't see any more questions it looks like our time is up but 395 00:35:49.190 --> 00:35:54.809 I would really like to thank the panel, again thank all of you and Maura and Robin, take 396 00:35:54.809 --> 00:35:55.539 us home. (Applause).